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ABSTRACT 


An  ambiguous  process  is  defined  as  a closed,  convex  set 
of  stochastic  processes  which  fulfills  certain  properties. 

In  the  context  of  computer  control,  ambiguous  processes  can 
be  used  as  simplified  models  for  the  complex  interaction  of 
a nonlinear  computer  controller  and  continuous  plant  with 
uncertain  dynamics.  Using  this  simplified  model,  and  applying 
results  on  worst-case  optimality  in  ambiguous  processes,  the 
feedback  system  can  be  better  understood  and  definite  per- 
formance bounds  can  be  calculated. 
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1.1  Finite-State  Control 

The  development  of  compact,  high-speed  and  inexpensive 
computers  over  the  past  several  decades  has  greatly  expanded 
the  range  of  applications  of  automatic  controls  (into  con- 
sumer electronics  and  medical  instruments,  for  example)  and 
has  made  possible  a vast  range  of  machinery  that  would 
otherwise  be  beyond  human  control,  such  as  rockets,  aircraft, 
and  complex  factory  and  plant  processes.  Concurrent  with 
the  development  of  the  necessary  hardware,  there  has  been 
increasing  understanding  of  the  interaction  between  control 
devices  and  systems,  i.e.,  of  feedback  control,  and  many 
principles  of  optimality  for  control  systems.  These 
capture  many  aspects  of  real-world  control  problems,  but  do 
not  take  into  account  the  actual  electronic  gadgetry  that  is  used 
to  implement  the  controller  systems.  In  some  cases,  computers 
have  sufficient  speed  and  precision  that  the  differences  between  the 
dynamics  of  the  actual  implementation  and  the  mathematical 
model  used  to  design  it  are  negligible.  But  when  we  push 
the  limits  of  computation  speed  and  accuracy,  it  becomes 
necessary  to  understand  the  differences  in  performance,  and 
at  present  there  is  little  theoretical  insight  on  this 
problem. 

Nevertheless,  the  very  fact  that  computers  are  capable 
of  high  speed  and  precision  points  out  an  obvious  issue: 


. i . 


could  computers  fcnd,  in  general,  the  interfaces  used  between 
them  and  the  continuous  system  being  regulated)  be  used  more 
efficiently  if  their  capabilities  were  accounted  for  theo- 
retically? Specifically,  could  we  get  by  with  less  equip- 
ment and  speed  than  we  are  now  using?  The  use  of  "optimal" 
control  theories  as  a basis  for  practical  design  may  result 
in  a complicated,  expensive,  or  costly  design  which  must  be 
simplified  if  it  is  to  be  implemented.  Engineers  are  de- 
finitely in  need  of  some  insight  into  the  interaction  of 
computer  and  real-world  systems  both  for  bounding  their 
performance  and  designing  cost-effective  control  solutions, 
and  predicting  performance  during  the  design  phase. 

Consider  the  following  example.  Suppose  we  wish  to 
balance  an  inverted  pendulum  using  state  feedback  from  the 
pendulum  angle  and  angular  velocity.  Torques  up  to  a cer- 
tain maximum  can  be  applied  to  the  pendulum  for  control. 

Modern  control  design  theory  would  first  linearize  the 
pendulum  dynamics  about  zero  and  find  the  optimal  linear 
state  feedback  map  u (t)  = Kx(t)  = k^9 (t)  + k2§ (t) . This 
would  then  be  implemented  by  digiting  9 (t)  with  an  A/D 
converter  to,  say,  eight  bits;  calculating  0 (t)  and  u(t)  in 
a microprocessor,  making  sure  the  resulting  control  does 
not  exceed  the  maximum,  and  converting  the  resulting  control 
level  back  to  analog  with  a DAC. 

For  the  application  at  hand,  though,  it  might  be  entirely 


satisfactory  to  use  a controller  of  the  following  kind: 

When  the  pendulum  velocity  exceeds  a certain  amount  ®max' 
controls  are  applied  to  reduce  the  predicted  velocity  to 
nearly  zero.  When  the  pendulum  angle  exceeds  a certain 
amount  0max'  again  a control  is  applied  to  put  the  predicted 
angle  near  zero.  Should  both  occur  simultaneously,  we 
first  reduce  the  velocity,  then  the  angle. 

This  control  law  calls  for  a compensator  with  two 
sequences  of  states,  probably  both  short,  which  apply  the 
proper  control  patterns.  The  only  input  this  compensator 
needs  is  one  of  three  levels  on  two  input  channels  (i.e.. 


- oo  to  -0  , 

max 


3 to  0 , to  ® on  the  0 channel) . It 

max  max  max 


could  be  very  easily  implemented,  and  should  more  smoothness 
in  the  control  be  desired,  the  control  sequences  could  be 
extended  as  necessary. 

The  major  problem  with  this  design  is  the  analysis  - to 
set  certain  criteria  for  its  performance . and  see  if  it 
fulfills  them.  Later  we  will  apply  rigorous  analysis  to 
this  example  and  actually  design  and  bound  the  performance 
of  a six-state  compensator. 

This  thesis  is  concerned  with  bounding 

the  performance  of  computer-based  control 


systems.  We  will  study  computer-based  control  in  its  own 
right,  letting  the  actual  limitations  of  the  computer 
dictate  the  direction  of  the  thoery,  as  well  as  the  actual 


structure  of  the  implementation-  To  this  end  we  start  with 
a very  general  model  of  a computer-based  controller,  not 
only  for  analytical  simplicity,  but  also  to  free  us  of 
artifically  imposed  constraints  of  present-day  computer 
architecture.  Specifically,  our  results  suggest  that  the 
add-divide-multiply  analysis  of  modern  digital  control  is  not 
essentially  necessary,  and  that  the  high-precision  data  acqui- 
sition A/D  converters  may  not  be  as  necessary  as  they  are 
assumed  to  be. 

Consider  this  very  general  model  of  a computer  control- 
ler: at  any  time  in  a discrete  time  set  T = 0,1,...,  the 

computer  receives  an  observation  of  the  system  state.  The 
resolution  of  this  observation  is  limited  by  a coder  (see 
Fig.  1)  which  can  only  distinguish  between  a finite  number 
of  possibilities.  Thus  the  observation  will  be  one  of  a 
finite  set  Y . Of  course  this  model  of  a coder  is  consistent 
with  present  day  A/D  converters,  but  is  much  more  general. 

Next  we  assume  that  the  computer  is  a finite-state 
machine  (refer  to  Minsky  [1] ) which  receives  inputs  from 
the  coder  and  changes  states  according  to  the  coder  observa- 
tion. Letting  Z be  the  finite  state  set,  there  is  a mapping 
x which  determines  the  controller  state  at  T+l  according  to 
the  state  at  T and  the  coder  observation  yeY  at  T: 

z (T+l ) = t(z(T) ,y (T) ) . 


L.  ' -j-  - - ■ . ...  -V.  - 1 


u 
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Figure  1. 


Finally  we  assume  that  the  controller's  output  to  the 
plant  at  time  T is  a function  of  the  coder  observation  at 
time  T and  the  controller  state  at  time  T.  Letting  u(T) 
represent  the  control  signal, 

u(T)  = o(z(T)  ,y(T)  ) . 

This  is  our  complete  model  of  a computer  compensator.  Any 
compensator  of  this  kind  will  always  be  referred  to  as  a 
finite  state  compensator  or  finite  state  controller. 

Sometimes  it  is  unrealistic  to  assume  no  delay  between 
the  coder  observation  and  the  determination  of  the  control, 
as  the  last  equation  suggests.  But  by  suitably  expanding 
the  state  space,  and  then  removing  the  direct  dependence  of 
the  control  signal  on  the  coder  observation,  any  delays 
introduced  by  the  coder  can  be  taken  into  account. 

1.2  Generality  of  the  Finite-State  Model 

The  compensator  described  in  the  last  section  is  not 
named  the  "finite  input,  output,  state"  compensator  because 
any  compensator  with  only  a finite  number  of  states  must  have 


this  structure  (under  only  one  other  assumption  which  is 
reasonable  for  digital  processing  equipment) . 

Any  deterministic  compensator  with  state  set  Z and  out- 
put set  U must  be  describable  by  two  equations 


z (T+l)  = r (x(T)  ,x(T) ) 


u = cr  (z  (T)  ,x  (T)  ) T = 0,1,.. 


where  z(T)eZ,  u(T)eU,  and  x(T)eX  is  the  state  of  the  plant, 
or  system  to  be  controlled,  at  time  T.  Look  at  the  map: 
t : Z x X -*■  Z. 

Since  Z is  finite  there  must  be  a finite  number  of  x- 
equivalent  regions  of  X defined  by 


'¥2 


= x ( z (z  ))  ; i.e.  x (z_  ,x)  = z„  iff  xeR 

qx  q2  q1  q2  c 


Since  the  number  of  regions  is  finite  and 

U Rz  z = x 
Z]eZ  Z1Z2 

z2eZ 

there  must  be  a minimal  finite  decomposition  of  X into  dis- 
joint regions  {R.},  i = l,...,r  such  that  each  R_  _ is 

1 zlz2 

contained  in  precisely  one  R^.  Define  the  map 

C:  X Y Y = (1, . . . ,r} 


so  that  C takes  xeX  to  the  index  of  the  region  R^  to  which  x 
belongs.  Call  C the  coder  associated  with  the  compensator, 
since 


for  some  t ' . 

We  have  to  make  one  additional  assumption  about  the 
compensator:  that  there  exists  a o'  such  that 

u (T)  = o'  (x  (T)  ,C(x(T)  ) ) 


This  assumption  is  realistic  for  digital  compensators  where 
there  is  no  direct  analog  feedthrough  from  the  input  to  the 
output,  so  we  see  that  x'  and  a'  and  Y satisfy  precisely 
the  requirements  set  down  for  a finite  compensator  as  in 
Section  1.1. 
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1.3  Summary  of  Past  Work 

Research  into  finite-state  control  has  been  somewhat 
scanty  because  of  the  difficulty  in  finding  a model  which  is 
both  theoretically  fractable  and  still  applicable  to  present 
problems.  Perhaps  the  most  successful  theory  relevant  to 
finite-state  control  systems  is  the  linear  discrete-time 
system  theory  (Freeman  [2])  especially  when  applied  to  linear 
algebraic  structures  such  as  rings  (Kalman,  Falb,  Arbib  [3] , 
Padulo,  Arbib  [4] , and  more  specifically  directed  toward 
control  is  Davis  [5] ) . Rings  over  the  integers  or  finite 
rings  most  closely  capture  the  finiteness  of  the  compensator. 
Unfortunately,  these  theories  are  not  useful  unless  a similar 
algebraic  structure  is  assumed  for  the  plant,  which  may  be 
unrealistic.  Some  attempts  have  been  made  at  hybrid  models, 
(discussed  generally  in  Kalman,  Falb,  Arbib  [6]  and  Johnson 
[7]  ) . 

Finite-state  control  is  found  in  the  optimal  solution 
of  the  general  minimum-time  problem  in  classical  control 
(Athans  and  Falb  [8]).  The  optimal  control  the  "bang  bang" 
controllers,  is  a finite-state  controller  in  the  sense  de- 
fined in  Section  1.1,  however  the  coder  is  highly  nonlinear 
and  in  general,  very  difficult  to  implement.  (In  the 
examples  of  finite-state  controller  cited  in  this  thesis,  the 
coder  will  always  divide  the  state-space  into  linear  regions, 
thus  simplifying  implementation.)  The  maximum  principle  has 
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been  extended  to  finite-state  systems  by  Sandell  [9]. 

In  this  thesis  we  will  also  be  considering  types  of 
uncertainty  that  are  not  probabilistic.  Much  past  work  has 
been  done  in  devising  alternatives  to  probability;  perhaps 
most  widely  known  is  "fuzzy  set"  theory  (Zadeh  [10]).  Our 
work  is  most  closely  related  to  the  "unknown  but  bounded" 
concept  discussed  in  Schweppe  [11] . 

1.4  Conclusions  of  Thesis 

In  this  short  work  we  have  not  been  able  to  explore  the 
approach  in  any  depth  and  so  we  can  only  draw  conclusions 
based  on  preliminary  theoretical  results  and  examples.  The 
major  areas  of  actual  results  are: 

1)  A method  by  which  complex,  nonlinear,  uncertain 
systems  can  be  simplified  to  finite  state  models  called 
"ambiguous  processes".  This  method  is  developed  specifically 
for  feedback  systems  composed  of  finite-state  compensators 
and  nonlinear  plants. 

2)  Properties  of  ambiguous  sytems  and  optimality  con- 
ditions form  which  performance  bounds  can  be  obtained. 

3)  Some  suggestions  for  designing  an  algorithm  which 
could  automatically  calculate  performance  bounds  for  a 
finite-state  control  system. 

The  qualitative  conclusions  of  this  thesis  which  we 
have  tried  to  support  with  mathematical  justification  are: 


15 


1.  Finite-state  compensation  is  sufficiently  complex 
that  the  most  useful  and  general  theories  will  study  bounds 
on  performance  rather  than  exact  performance.  We  have  taken 
this  approach  in  our  work  and  shown  that  methods  of  attack 
which  are  not  guaranteed  to  deliver  exact  measures  but  may 
guarantee  worst  case  performance  bounds  can  give  significant 
insight  into  a finite-state  feedback  system. 

2.  Through  examples  we  will  show  that  simple  finite- 
state  machines  with  only  a few  inputs,  outputs  and  states 
can  compensate  a system,  which  under  present  theories  would 
require  a more  complex  design.  Furthermore  we  will  show  how 
to  calculate  a lower  bound  on  the  performance  of  such  a com- 
pensator. 

3.  We  also,  conclude  that  some  ideas  from  artificial 
intelligence  car.  fcf>  made  rigorous  and  applied  to  problems 
in  finite-state  compensation.  This  will  become  apparent  in 
the  sequel  as  we  introduce  knowledge  spaces,  conceptual 
states  and  discussion  of  optimizing  control  given  a par- 
ticular conceptualization  of  the  problem. 

1.5  Overview  of  Methods  and  Results  - 
Bounding  Performance 

To  illustrate  the  methods  and  results  formalized  in 
subsequent  sections,  we  will  show  their  application  to 
simple  examples.  The  discussion  here  will  be  informal  and 
intuitive,  so  the  reader  can  more  easily  understand  the 
motivation  for  the  formal  mathematical  definitions  to  be 

I 


1 


16 

given  later. 

Consider  first  the  problem  of  bounding  the  performance 
of  a given  compensator.  We  assume  the  plant  is  defined  in 
terms  of  a finite-dimensional  Euclidean  state  space  X = Re- 
state transition  function  f,  which  takes  the  discrete-time  state 
x(T)eR  , control  u(T)eRm,  and  noise  value  v(T)eRp,  to  the 
next  state 

x (T+l)  = f (x(T)  ,u(T)  ,v(T)  ) 

and  sets  of  possible  probability  measures  for  the  initial 
state  and  for  the  noise  occurrances  v(T);  these  sets  could  contain 
for  example,  only  a single  probability  measure,  corresponding  to 
a stochastic  system.  Regardless  of  whether  the  measure 
sets  for  x(0)  or  v(T)  contain  more  than  one  probability 
measure,  our  analysis  will  necessarily  lead  to  measure  sets 
with  more  than  one,  so  it  is  convenient  to  make  the  more 
general  assumption  from  the  beginning.  For  practical  cal- 
culations we  will  introduce  the  simplical  measure  sets  in 
Chapter  3 which  are  finitely  describable  and  simplify  cal- 
culations. However,  since  the  formal  development  is  some- 
what involved.  Chapter  2 will  proceed  to  develop  the  major 
ideas  assuming  only  the  minimal  necessary  properties  of  the 
measure  sets  for  x(0)  and  u(T).  A system  of  measure  sets 
with  these  properties  will  be  dubbed  an  "ambiguous  measure  space". 
Simplical  measure  sets  cannot  describe  an  arbitrary  simple 
probability  measure,  so  in  practical  examples,  if  the  model 

Lfc i i i - — ■ 
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of  the  plant  includes  a probabilistic  noise  model,  then  the 
probability  distribution  of  x(0)  and  v(T)  will  have  to  be 
enclosed  in  a simplical  measure  set.  Since  the  true  distri- 
bution is  somewhere  in  this  set,  the  true  performance  must 
be  within  the  bounds  calculated,  but  better  bounds  would  of 
course  be  obtainable  if  the  actual  probability  were  known. 

There  are  however,  some  additional  advantages  to  the 
simplical  measure  set  model.  First,  some  kinds  of  knowledge 
about  the  plant  or  the  noise  can  be  best  captured  by  a set 
of  probability  measures.  For  instance,  knowing  0 x <_  2 
could  be  restated  that  the  possible  probability  measures  for 
x must  have  all  mass  in  the  interval  [0,2].  This  gives  a 
possible'  set  of  measures,  rather  than  a single  measure,  to 
represent  the  knowledge  about  x,  and  indeed  this  set  is  a 
simplical  measure  set.  This  idea  of  representing  noise  only 
by  a set  of  possible  values,  or  the  unknown  but  bounded 
noise  model,  has  been  formulated  elsewhere  (Schweppe  [II  ]) 
but  has  not  been  developed  too  far  in  control. 

Second,  there  are  special  cases  where  the  optimal 
finite-state  compensator,  given  only  a simplical  measure  set 
as  a priori  information,  may  indeed  be  optimal  over  all 
compensators.  This  relies  on  a property  called  finite 
information  lifetime.  ' 

Whatever  the  noise  model  chosen,  probabilistic,  simplex 
measure  set,  etc. , the  dynamics  of  any  computer  feedback 


control  are  complex  and  difficult  to  analyze  due  to  the 
thresholding  effects  of  the  coder,  and  finite  memory  of  the 
computer.  To  simplify  the  feedback  system  sufficiently  for 
analysis,  we  model  it  as  an  ambiguous  process , from  which 
definite  bounds  on  the  performance,  although  not  the  exact 
performance  can  be  obtained.  An  ambiguous  process  is  a set 
of  non-stationary  Markov  process  whose  transition  prob- 
abilities are  known  to  lie  within  some  range;  the  ambiguity 
arises  from  simplifying  the  dynamics  of  the  system  to  a 
finite-state  model.  To  make  these  ideas  more  concrete, 
consider  the  following  first  order  system: 

x (T+l)  = | x(T)  + u(T)  + v (T)  . 

Let  x(0)  be  a random  variable  with  all  mass  between  -1  and  1, 
and  let  v(T)  be  a random  variable  with  mass  .1  in  (-j,  -j) , 

.4  in  (.-j,  0),  .4  in  (0,  ^)  , and  mass  .4  in  (j,  ^0  , for  all 
T.  We  take  open  intervals  to  avoid  intersecting  sets  re- 
sulting in  a single  point.  We  represent  the  a priori  know- 
ledge of  x(0)  and  v(T),  then  by  the  diagrams  in  Figure  2. 


x(o) 


•i  .t. 


vro,T=o, 


Figure  2. 


L 
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It  has  been  suggested  to  control  this  unstable  system 
with  a single-state  compensator  with  Z=  {z},  U=R, 


y = C (x) 


-i  < x < 1 
1 < x < 3 
if  |x|  > 3 


t (z,y)  = z Vy  , 


ct  ( z , y ) =/+2  y = 1 

) o y = 2 
/- 2 y=3 

\ 0 y = o 


The  coder  value  of  zero  represents  failure  of  the 
sysbem,  i.e.,  if  the  system  state  ever  exceeds  3 or  -3,  we 
are  no  longer  interested  in  the  behavior.  Diagramatically 
we  represent  the  compensator  in  Figure  3. 


2/0 

1/+2 

3/-2 


Figure  3. 

Finally  take  as  a performance  criterion  of  this 


system 

j = - * if  | x (T ) | > 3 for  any  T = 0 , . , . . . 


J = lim  inf  ^ ~[x(T)]2 

n=0 


— 


■k 
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To  place  bounds  on  J we  use  a conceptualization  of  the 
closed  loop  system  which  neglects  some  of  the  details  of  the 
exact  dynamics,  which  gives  rise  to  an  ambiguous  process 
model.  As  the  simplest  conceptualization  we  decompose 
the  closed-loop  state  space  into  four  regions : 

XxZ  = u C-1  (y) x{ z} 
yeY 

call  these  regions  Aq,  A^,...,A3-  At  any  time  T = 0,1,... 

(x  z)  is  in  exactly  one  of  these  regions,  and  there  are 
probabilities  associated  with  each  possible  transition  to 
another  A^  in  the  next  step.  Because  the  probability  measure 
of  u(T)  is  not  known,  and  because  the  probability  measure  of 
x(T)  is  not  known,  but  is  only  known  to  lie  within  some  A^ , the 
exact  probability  for  the  transition  to  A^  at  time  T+l 
cannot  be  determined.  However,  a set  of  possible  probabilities 
can  be  determined.  The  possible  probabilities  can  be  used 
ultimately  to  limit  the  performance  J. 

Consider  for  example,  x(T)eC-1(2);  i.e.,  x(T)e[-l,l], 

3 3 

and  x(T+l)  = ^ x(T)  + v(T).  j x(T)  must  lie  in  the  range 
[~|,  ^-]  . One  of  the  possible  measures  for  j x(T)  is  6 ^ , 


the  measure  with  unit  mass  at  -j  . Thus  a subset  of 
possible  measures  for  x(T+l)  is 

r w . .1.1.1 -I 

03  *{y)  ^ < M * i t 1 

2 “I  0 


where  {y}  is  the  set  of  all  possible  measures  for  v(T). 

Examining  the  diagram  we  see  that  for  6 3 the  probability 

~2 

of  going  from  A2  to  A^  can  be  as  high  as  1.  By  symmetry  we 
see  that  the  probability  of  going  from  A^  to  A^  can  also  be 
1.  By  taking  x(T)  v <5q,  we  see  that  the  probability  of 
staying  in  A2  could  be  as  high  as  1.  We  therefore  conclude 
that  the  transition  probabilities  to  Aq,  A^ , A 2,  A^ 
starting  from  A2  is  a vector  (we  differ  here  from  the 
standard  notation) 


P20> 

p20eC0] 

P21 

P21e  [0,1] 

where 

p22 

P22e [0,1] 

p23 

p23e[0'1] 

and  (1,1, 


1)p2x 


1.  For  convenience  we  write  this  as 
’[0] 

[0,1] 
p2xe  [0,1] 

[0,1] 


with,  the  understanding  that  p2x  is  a probability  vector  so 
only  choices  with  (.1, 1, . . . , l)P2x  valid.  The  reader  can 
check  that 


*Jm. 


22 


p.  e 
^lx 


r[0]  ' 

f 

[0] 

10,1] 

[0] 

[0,1] 

P3x  = 

[0,1] 

, 

[0,1] 

k 4 

We  now  arrange  the  vector  into  a matrix  which  represents 
the  transition  matrices  possible: 


no] 

[0] 

[0]  > 

[0,1] 

[0,1] 

[0] 

p ' 

' 

[0,1] 

[0,1] 

[0,1] 

• 

k [0] 

[0,1] 

[0,1]J 

Seeing 

that 

the 

top  row  must  be  zero 

, the  closed-loop 

can  therefore  never  fail  so 

long  as 

x(0)  is  in  A^,  A2 

Thus  J 

> — OO 

and 

we 

concern 

ourselves 

with  the  matrix 

r[o,i] 

[0,1] 

[0]  " 

P 

= 

[0,1] 

[0,1] 

[0,1] 

[0] 

[0,1] 

[0,1] 

which  we  take  as  defining  an  ambiguous  process,  and  it  is 
possible  for  this  somewhat  trivial  model,  that  the  closed 
loop  state  could  remain  in  A^  and  A^  forever,  and  if 
(x  zJeAj^  or  A3,  x can  be  as  large  as  3,  so  J could  be  as 


small  as  -9,  since 

T 


1 t 2 i T 

J = lim  inf  T -x  >_  lim  inf  T £ 

T-*»  n=l  T-v®  n=l 


1 


, 


-9 


-9 


This  is  not  a very  good  bound  so  we  will  try  a more 
detailed  coneptualization.  Denote  the  knowledge  that  x(T)eA2 
by  2 ; this  will  be  called  a generalized  state . In  our 
previous  conceptualization  the  closed-loop  system  was  repre- 
sented by  Q’1] 

fn,l 


Now  let  (n;  denote  the  generalized  state  of  the  closed 
-loop  system  of  T if  x(T-l)eA2  and  x(T)eA^.  We  expand  our 
conceptualyation  to : 


where  we  have  duplicated  the  © on  the  right  side  for 
graphical  purposes,  and  the  arrows  represent  the  possible 
transitions  that  can  occur.  We  now  place  bounds  on  the 
probability  of  each  possible  transition. 

The  transition  from  © can  be  arbitrary  as  we  have 
previously  seen.  However,  the  probabilities  of  the  various 
transition  leaving  (21)  are  different  from  the  ones  calculat 


ed  previously  for  the  more  general  conceptual  state 
since  we  have,  in  addition  to  the  information  that  x(t)eA^, 


the  information  that  x(T-l)eA 


Thus  we  can  limit  the 


To  see  this,  notice  that  all  of  the  mass  of  any  measure 

3 

of  x,  given  21  , must  lie  in  [-2,-1],  so  that  j x(T+l)  + u(T) 
must  lie  in  [-1,  j]  , and  x(T+l)  + u(T)  + v(T)  must  have  all 

3 

mass  in  [-■ j,  1] . But  notice  that  the  greatest  probability 
that  x(T+l)eA1  is  .5  and  occurs  when  x(T+l)  + u(T)  v 6_^. 

The  smallest  probability  for  this  transition  is  0.  If  in 
fact  x(T+1)eA1  then  our  knowledge  about  x(T+l)  is  211  , and 
x(T+l)  must  lie  in  [-j,  -1],  so  x(T+2)  will  surely  lie  in 
A2*  Using  this  information  we  can  label  each  transition 
with  its  set  of  possible  probabilities: 


We  now  ask  what  is  the  minimum  J possible  given  this 
range  of  transition  probabilities  at  each  step,  and  also 
knowing  that  x(0)eA2.  Intuitively  we  see  that  the  lowest 
possible  return  will  occur  by  taking  the  probability  to  211 
and  233  as  high  as  possible,  so  to  obtain  a lower  bound  on 
J we  calculate  the  limiting  ergodic  state  probabilities  of 
the  worst  case: 


The  limiting  state  probabilities  p must  satisfy: 


0 .5  1 

1 0 0 p = p 

0 .5  0 


Pm  = 2/5 


J = lim  inf  T 2 -x2  > J (-1)  + \ (-9)  + i (-9)  * - 

T->-<»  n-1  — 5 5 5 

which  is  a considerably  improved  bound. 

The  two  conceptual •*  ions  of  the  feedback  system  des- 
cribed above  gave  rise  to  two  ambiguous  processes.  One  of 
the  major  results  of  this  thesis  is  to  prove  that  a lower 
bound  on  the  performance  of  an  ambiguous  process  is  the 


result  of  some  choice  of  the  transition  probabilities  at 


their  extreme  points , and  that  this  is  a valid  lower  bound 
even  if  the  real  transition  probabilities  change  at  each 


By  increasing  the  number  of  generalized  states,  better 
and  better  bounds  can  be  obtained.  Unfortunately  we  have 
not  been  able  to  prove  that  these  bounds  must  approach  the 
true  performance,  although  it  seems  very  likely.  We  have 
though,  formalized  these  ideas  and  found  a number  of  opti- 
mality conditions  which  could  lead  to  a bounding  algorithm. 
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FINITE-STATE  COMPENSATION  AND  AMBIGUOUS 
PROCESSES-THEORY 


2.1  Most  of  the  ideas  that  were  discussed  informally  in  the 
Introduction  and  Overview  will  be  nailed  down  precisely  in 
this  Chapter.  Conceptually,  the  material  divides  into  two 
parts.  In  the  first  part,  encompassing  Sections  2.2  through 
2.7,  a precise  model  of  a feedback  control  system  involving 
a finite-state  compensator  and  continuous  plant  is  defined 
section  by  section,  and  it  is  shown  that  any  such  feedback 
system  can  be  simplified  to  a finite-state  model  called  an 
ambiguous  process . The  name  arises  because  the  transition 
probabilities  for  the  simplified  finite-state  model  may  not 
be  well-defined. 

The  second  part,  Sections  2.8  through  2.12,  develops  an 
abstract  theory  of  ambiguous  processes,  without  making  ref- 
erence to  their  application  in  finite-state  control. 
Optimality  conditions  are  derived  and  a fundamental  bounding 
theorem  is  proved.  This  theory  will  form  the  basis  for 
actual  methods  of  bounding  the  performance  of  compensators, 
which  will  be  taken  up  in  Chapter  3. 


2.2  Ambiguous  Measure  Spaces 

In  formulating  a model  of  a compensated  plant  it  is 
reasonable  to  expect  some  uncertainty  in  the  behavior,  even 
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if  the  control  inputs  are  precisely  known.  This  uncertainty 
can  be  modeled  probablistically , but  for  our  purposes  later 
on  it  is  convenient  to  model  uncertainty  in  a more  general 
way,  which  includes  probability  as  a special  case.  In  fact, 
even  if  the  problem  is  formulated  probabilistically,  this 
more  general  notion  of  uncertainty  arises  naturally  in  our 
analysis.  To  see  how  this  comes  about,  we  outline  briefly 
the  major  steps  in  reducing  a compensated  system  to  an 
ambiguous  process. 

Suppose  that  a compensated  system  is  running,  and  we, 
as  an  outside  observer,  make  observations  of  the  progress 
of  plant  state  x(T)  and  the  compensator  state  z (T) . The 
compensator  state  is  always  an  element  of  the  finite  set  Z 
(as  defined  in  Section  1.2)  so  we  will  assume  that  the 
compensator  state  is  always  observed  perfectly.  On  the 
other  hand,  the  plant  state,  x(T),  lies  in  Euclidean  space, 
and  we  will  assume  that  only  finite  information  about  x(T) 
is  observed;  i.e.,  there  is  a finite  number  of  "areas"  of 
the  plant  state  space,  and  only  the  area  of  x(T)  is  observed. 
Just  knowing  that  x(T)  lies  in  some  area  does  not  imply  a 
unique  probability  measure  for  x(T),  unless  the  area  consists 

litS 

of  a single  point,  so  the  knowledge  that  x(T)Nin  some  area 
is  not  probabilistic  knowledge  in  and  of  itself.  All  that 
can  be  said  is  that  if  x(T)  is  measurable,  all  probability  of 
that  measure  must  lie  in  the  observed  area. 


Suppose  In  addition  to  our  knowledge  that  x(.T)  lies  in 
some  area  A^,  which  we  will  call  an  immediate  area,  we  also 
know  that  x(.T-l)  lay  in  some  immediate  area  A2*  Then  it 
might  be  possible  to  rule  out  some  potential  probability 
measure  for  x(Tl  just  knowing  that  x(T-l)eA2.  Thus  we 
need  to  represent  certain  kinds  of  measure  sets  in  our 
analysis. 

The  first  step  in  reducing  a compensated  system  to  an 
ambiguous  process  is  to  define  these  immediate  areas  and 
the  knowledge  associated  with  each.  To  give  some  hint  of 
future  rotation,  £ will  represent  the  "know  nothing"  un- 
certainty, and  it  will  be  defined  as  the  set  of  all  prob- 
ability measures  on  X = Rn.  To  represent  the  knowledge 
that  x(T)eA^,  we  need  a projection  operator  which  zeros  any 
probability  outside  the  known  area;  we  will  write 


where  ru  / is  the  knowledge  associated  with  observation  A. . 
A1  1 

After  these  areas  have  been  defined,  the  second  step  is 

to  define  a set  of  histories  - a set  of  recent  observation! 

which  the  observer  is  assumed  to  be  able  to  remember.  A 

history,  call  it  might  be  that  x(T-n)eAn,  x (T-n+1)  eAn_^ , 

...,x(T)eA  . and  that  z (T-n)  = z ....  . Again  z (T)  = z . 

o n o 

We  can  associate  with  each  history  tt  some  (non-probabilistic) 
knowledge  about  x (T) ; for  our  knowledge  about  x(T-n)  was,  in 


30 


We  must  be  somewhat  vague  here  because  the  development  of 
these  concepts  rigorously  will  be  a lengthy  process.  Suffice 
it  to  say  that  corresponding  to  each  history  is  a set  of 
possible  probability  measures  on  x(T). 

The  third  step  in  defining  an  ambiguous  process  will  be 
to  recognize  that  the  observed  histories  form  a sequence  in 
time,  call  it  it  (0)  , tt  (1)  , . . . . A journal  will  be  difined 

as  an  automaton  whose  state  space  is  the  set  of  possible 
histories,  and  which  makes  transitions  from  one  history  to 
the  next  depending  on  each  observation.  It  is  clear  that 
there  is  a "best”  journal  for  each  set  of  histories,  namely 
the  journal  that  carries  the  greatest  amount  of  information 
from  past  histories  into  future  histories,  within  the 
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limitations  of  its  finite  state  space. 

The  last  step  is  recognizing  that  the  journal  is  an 
"ambiguous  process",  in  the  following  sense.  The  probability 
of  making  each  observation  at  some  time  T+l,  (which 
determines  the  probability  of  going  to  each  new  journal 
state,  it  (T+l))  depends  on  the  probability  measure  of  x(T). 
Associated  with  the  present  journal  state  tt  (T)  is  some 
knowledge  about  x(T),  as  we  described  in  the  third  step. 

Hence  there  is  t.ome  "ambiguous"  probability  for  each  transi- 
tion to  the  next  journal  state,  based  on  the  knowledge  of 
x (T)  that  can  be  derived  from  it  (T)  . 

The  development  of  the  above  ideas  on  a rigorous  basis 
is  a task  that  lays  before  us  in  the  next  several  sections. 

It  is  clear  that  a generalized  notion  of  uncertainty  is 
pervasive  in  the  analysis.  Therefore  we  begin  in  this 
section  by  defining  the  concept  of  an  ambiguous  measure 
space. 

Definition:  A probability  measure  y over  Rn  is  a 

n '■)  w 

positive  Borel  measure  with  y (R  ) = 1.  will  always  denote 
the  Borel  sets  in  Rn. 

Note:  Any  measure  will  always  be  a probability  measure 

unless  otherwise  stated. 

Definition:  An  ambiguous  measure  space  N over  Rn  is  a 

collection  of  measure  sets  (of  probability  measures)  over  Rn 


with  the  following  properties: 


I 
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a)  £n  = set  of  all  probability  measures  in  Rn,  is  in 

N. 

b)  xeiRn  ■+■  {5  }eN  (6  is  measure  with  unit  mass  at  x)  . 

A X 

c)  A set  S is  representable  in  N iff 

{y|y(S)  = 1,  y (S)  = 0}eN. 

Define  for  any  set  Se  and  any  probability  measure  y 
with  u(S)  0 a new  measure 

y(s,«s) 

p4<li)  (si>  - — U(sr  a11  • 

If  neN  is  a measure  set,  let 

Ps(n)  = (Pg(y)|u(S)  ? 0,  yen} 

We  require  that  Pg(n)eN  for  every  representable  S,neN. 
Ambiguous  measure  spaces  are  useful  for  modeling  many 
kinds  cf  uncertainty.  Here  are  some  examples. 

Example  1.  (Ordinary  Probability)  Let  X = <Rn  and  let 

= { { y > | y is  a probability  measure  on  C?)  . 

Let  (S~  be  the  set  of  nonempty  Borel  sets  in  Pn.  Let  Be  & , 
and  define: 

<?g  = {y  1 y is  a probability  measure  on  (8n  and 
y(B)  = 1} 

n2  = | Be  0" } . 

, the  set  of  all  probability  measures,  is  not  an  ambiguous 
measure  space  because  properties  (b)  and  (c)  do  not  hold. 
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However,  N = N^UN2  is  an  ambiguous  measure  space,  and  we 
check  the  necessary  properties: 

a)  £neN  since  Rn  is  a Borel  set  and  gn  = 5rn 

* Rn 

b)  { 5 } eN  for  every  xeiRn  since  {x}efef  and  6 is  a 
probability  measure,  hence  {6  }eN  , {5  }eN. 

X J.  X 

c)  If  neN  either  neN.^  or  neN2  (or  both)  consider 
first  neN^.  Then  n = {y}  and  let  S be  a representable 
set;  then  either  y (S)  =0  or  y (S)  ^ 0.  If  y (S)  = 0,  then 
Pg(n)  = <f>eN;  and  since  the  representable  sets  are  just 
the  Borel  sets,  y (S)  ^0  implies  Pg(ri)  is  the  set  of  the 
single  measure  defined  by 


y*  (S') 


u(sns') 
y (s) 


which  is  also  in  N. 

Now  consider  the  case  where  neN,,.  Since 


VO  - 


» 


%n 

’sns 


if  BOS  = 0 
otherwise 


we  conclude  that  Pg(n)eN,  therefore  Pg(n)eN  for  every 
representable  S,  and  N is  ambiguous  measure  space. 


neN, 

□ 


Note  that  Example  1 is  the  smallest  ambiguous  measure 
space  which  contains  all  the  sets  of  a single  probability 
measure. 


Examp le  2 . 


(Trivial  Ambiguous  Space) 


N = }U{  { 6 } :XERn} 

Rn  x 


- | ' | - | M M I I i 
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is  the  smallest  ambiguous  measure  space.  The  knowledge 
that  an  unknown  that  is  represented  by  an  neN  is  either 
"the  unknown  is  equal  to  x"  or  "the  unknown  could  be  any- 


thing in  Rn  with  any  probability  measure". 


□ 


Example  3 . Let  X = R and  define  ri  • ^ 0 < p < 1,  as 

P 

n = (y|y  is  a measure  on  R,  y(-®,0)  = 1-p  , 

^[0,®)  = p} 

The  knowledge  we  would  have  about  an  uncertain  number  x 
given  rip  is  that  there  is  a probability  p that  x >_  0 . This 
kind  of  knowledge  cannot  be  represented  by  a single  prob- 
ability measure. 

Under  our  definition,  though,  (n  |0<p<l}  does  not 

pi  - r - 

constitute  an  ambiguous  measure  space . 


□ 


Example  4 . Let  S = Rn  and  let  be  the  set  of  all  nonempty 
Borel  sets  in  Rn.  Let 

nB  = (y|y  is  a measure  on  Rn,  y (B)  = 1 } , b e (f  . 


It  can  be  verified  that  the  set 

(nB,  , Be©  } 


does  satisfy  all  the  axioms  of  an  ambiguous  measure  space. 


Example  5.  Suppose  we  know  that  xeRn  and  that  x is  a 


random  variable.  The  distribution  has  a continuous  density 
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and  it  is  known  that  the  median  is  somewhere  between  -1  and 
1 inclusive.  It  is  interesting  that  this  set  of  possible 
measures  on  x can  be  reduced  to  the  kind  of  description 
given  in  Example  1. 

Let 

= {p  | p (— <»,  — 1]  = .5,  y[-l,»)  = .5 
n2  = {p  | p (-<==,1]  = .5,  y[l,°°)  = .5 
n3  = {y |u [-1,1]  = 1} 

It  will  be  noted  here  without  proof  that  the  measure  set  of 
x is  contained  in  the  convex  hull  of  n2  n2.  In  fact, 
it  is  dense  in  the  convex  hull. 


□ 


In  the  next  chapter  we  will  define  the  simplical 
measure  space,  that  includes  all  the  measure  sets  of 
Examples  (1)  and  (3) , and  all  those  on  Example  (2)  when  B 
is  a simplex.  For  the  moment,  we  proceed  in  a more  general 
framework.  Consider,  for  example,  the  biggest  ambiguous 
measure  space: 

Definition : is  the  set  of  all  subsets  of  probability 

measures  on  Rn. 

Obviously  is  too  general  for  any  computational  pur- 
poses, but  it  is  useful  to  conceptualize  for  general 
theoretical  results  later  on. 


36 


2.3  Definition  of  the  Closed-Loop  System 

Having  the  definition  of  ambiguous  measure  space  be- 
hind us,  we  can  now  formulate  the  problem  in  a generalized 
uncertainty  model  which  is,  as  we  described  earlier,  per- 
fectly natural  and  amenable  to  our  line  of  analysis,  and 
in  addition,  useful  and  even  more  convenient  than  regular 
probability  for  some  of  the  bounding  methods  discussed  in 
Chapter  3. 

The  general  idea  is  that  the  closed-loop  system  is  a 
non-stationary  Markov  process  on  the  product  space  XxZ , 
(defined  in  Section  1.2)  and  at  each  time  T the  noise  is  a 
random  variable  v(T)  having  a distribution  in  some  measure 
set  n , and  the  state  has  an  initial  distribution  in  some 
measure  set  nx«  We  do  not  however,  know  which  of  many 
possible  Markov  processes  occurs,  since  we  do  not  know 
which  probability  measure  actually  describes  v(T)  and  x(0). 
However,  for  any  performance  criterion,  a maximum  and 
minimum  over  this  possible  set  of  Markov  processes  can  be 
defined. 

In  the  next  two  sections  we  will  concentrate  on  a pre- 
cise definition  of  these  concepts  before  launching  into  the 
development  of  ambiguous  processes  derived  from  the  actual 
closed-loop  system. 


Let  us  review  the  notation  introduced  in  Section  1.2: 
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X = Rn 
Z 

X X Z 

U C Rm 
RP 

f:  X x U x RP 
T'  : X X Z -*•  Z 
a:  X X Z U 
Y 

C : X Y 

T*  : Y z Z -*■  Z 

o':  Y x Z -»■  U 


plant  state  space 

compensator  state  space,  q elements 
closed  loop  state  space 
admissable  control  set 
admissable  noise  set 
X plant  state  transition  map 
compensator  transition  map 
compensator  output  map 
coder  equivalence  classes,  r elements 
Canonical  coder  map 
canonical  state  transition  map 
canonical  compensator  output  map 


Table  1. 


(We  may  omit  the  primes  on  x'  and  o'  when  clear  from  context.) 

Now  let  N be  a ambiguous  measure  space  over  Rp  and  let 
N be  a knowledge  space  over  Rn.  Assume  that  the  set  of  all 

X 

possible  probability  measures  of  the  noise  v(T),  T = 0,1,... 
is  a measure  set  nveNv.  Similiarly  assume  that  the  set  of  all 
possible  probability  measures  of  x(0)  is  a measure  set 
ri0£Nx‘  We  coul<3  have»  for  example,  both  nv  and  n0  consisting 
of  a single  probability  law  but  this  is  sometimes  a clumsy 
case  for  actually  computing  bounds,  as  we  shall  see  in  Chap- 
ter 3.  It  is  easier  to  have  nn  and  n simplex  measure  sets, 
or  equivalently,  N and  N simplex  knowledge  spaces,  as 

X V 
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described  in  the  next  Chapter. 

Next  assume  that  C ^ (y)  is  representable  in  N for  all 

X 

yeY,  so  that  C 1 (y)  is  a Borel  set  and  we  can  define  the 
conditional  probability 

v [xeC_1 (y) ] (B)  = y[Enc-1(y)]  BeBn 

Our  last  assumption  is  on  the  smoothness  of  f.  For  any 
Xenv,  ueU,  and  y,  a probability  measure  on  X,  we  require  that 

QU(B)  = { (x,y)  | f (x,u,y) eB}  (BeBn) 


be  measurable  in  the  product  measure  y x X;  i.e.,  Q(B)  is  a 
Borel  set  in  Rnp.  Then  Fubini's  theorem  guarantees  that  we 
can  assign  a unique  measure  to  Q(B) 


J d(y  x X)  = J X{y| (x,y)£Qu(B) }dy 

(Rn 

= J y{x | (x,y) cQu (B) }dX 


Qu(B) 


n 


£R 

It  can  be  verified  that  y' (B) 


■/ 

QU<B) 


d (y  x X ) (BeBn)  is 


another  probability  measure  on  the  Borel  sets  of  Rn,  so  define 
the  mapping 

Fu : pm(X)  pm(Rp)  -*■  pm(X) 


where  pm(X)  is  the  set  of  all  probability  measures  on  X,  by 


Fu(y,X)  = J d(y  x X) 


Q (B) 
u 


which  represents  the  new  probability  measure  on  X after 
applying  control  u,  given  that  the  previous  measure  of  X was 


Proposition.  If  f is  linear  then  F exists. 

— 

We  can  now  define  the  set  of  Markov  processes  described 

in  the  beginning  of  this  section.  At  each  time  T the  closed 

loop  state  (x (T) , z (T) ) is  in  the  set  X x Z.  Letting  & be 

zi 

the  vector  in  R-  with  1 at  position  i and  zero  elsewhere,  the 
probability  measure  at  T = 0 of  (x(0),z(0))  is: 


“o  x s2(0) 


(T  =.  0,  uocn0) 


for  some  u en  . 

o o 


Suppose  that  the  actual  probability  measures  y , 


X (0)  ,X  (1)  , . . . env  for  the  initial  state  and  noise  probabili- 


ties were  known.  Call  this  a Markov  relization  of  the  closed 


loop  system.  Then  each  Markov  realization  ({X(T)},y  ) will 


indeed  be  a Markov  process  with  probability  measures  on  the 
product  space  for  T = 0,1,... 

{ (y  x p)T} 


T=0 , 1 , . . . 


if  we  define 


(U  X p)o  - „o  X «I(0) 


and 


(B  X p)  = £ 

ycY 


Urp  (y ) 2,y)^T^X^^  (y)  ) fXT)  x 6,p  1 u)J  ] 


zeZ 


yT(y) >o 
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for  T = 1,2,...  . It  is  clear  from  our  definition  of 

(y  x P)T+1  that  the  closed  loop  state  will  be  a Markov  process 
since  the  probability  measure  of  (y  x P)T+^  depends  only  on 
the  probability  measure  of  (y  x p)T. 

We  now  define  the  closed  loop  system  as  the  set  of  all 
probability  sequences 

| { (y  x P)T>T=0fi, . . . (2) 

defined  from  the  above  equation  (1)  for  all  Markov  realizations 
(yo,U(T)}). 

Also  for  each  Markov  realization  let  {x  (0) ,x (1) , . . . } be 
the  random  plant  state  and  (z(0),z(l),...}  be  the  random 
compensation  state.  Each  set  is  a stochastic  process  but 
either  taken  alone  is  not  in  general  a Markov  process,  since 
the  statistics  of  the  next  compensator  or  plant  state  is 
determined  by  both  the  previous  compensator  state  and  plant 
state. 

2 . 4 Measures  of  Performance 

To  measure  the  performance  of  a compensator  we  will 
assume  that  a certain  cost  or  penalty  is  expended  at  each 
point  in  time, (maybe  zero),  which  depends  on  the  plant  state 
x (T) . Let 

R:  X -*■  [ 0 , 00 ] 


be  the  mapping  of  states  into  appropriate  costs.  A cost  of 
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infinity  in  some  state  be  considered  a "failure"  or 
"catastrophe"  at  that  time;  any  such  state  must  be  anticipated 
and  avoided  regardless  of  the  cost  accumulated  on  doing  so. 

Of  course,  R can  be  defined  in  the  range  [0,°°)  if  immediate 
failure  is  not  a possibility.  We  make  the  following  assump- 
tions about  C; 

1)  Let  F be  the  set  of  xeX  such  that  R(x)  = °°  . (F  may 
be  empty.)  Then  F is  representable  in  Nx. 

2)  sup{C  (x)  :x/F}  < =°. 

Now  we  take,  as  our  performance  criterion,  the  highest 

average  cost  that  could  occur  which  we  denote  J.  To  make 

this  precise,  consider  the  random  variable 

T 

y = lim  sup  i £ R(x(T)) 

T-h»  t=0 


of  a particular  Markov  realization  ({X(T)},y  ). 

The  probability  mass  of  y must  lie  within  the  interval 
[ 0 , °° ] , and  we  denote  the  maximum  cost  from  a Markov  realiza- 
tion, Ji  , = inf {a:  Pr(y  < a)  = 1}  . We  can  then  define 

1 a j , y 


the  maximum  average  cost 


J = sup 

{X (T) }Cn 


(X},y 


yoEno 


From  this  definition  we  conclude  that: 

1)  J = oo  iff  the  system  can  fail,  or  more  precisely, 
iff  for  some  realization,  x(T)eF  with  nonzero  probability  for 
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irr-r 


some  T > 0 ; 


2)  lim  sup  T ^ 
t=0 


. T 1 

1 R (x  (T) ) < J with  probability  one  for 


X->oo 


for  realization,  and  there  is  a nonzero  probability  that 

T 

J - e <_  lim  sup  T ^ R(x(t))  £ J 

t=0 


for  some  realizations  for  everv  e > 0. 


Proof : These  facts  follow  easily  from  the  definitions. 

We  will  prove  (1) . Assume  there  is  zero  probability  that 


x(t)eF  for  every  t ^ 0,  every  realization  { X (T)  } , y . Then 


every  J'  is  Lo^nded  above  by  sup  { R(x):  xjafF},  which  by 
assumption  is  not  so  J is  also  bounded  from  above. 


2 . 5 Example : Tracking  and  Regulation 

As  an  example  of  a possible  performance  criterion, 
suppose  one  wants  to  regulate  or  track  or  compensate  against 
some  unknown  disturbances,  and  it  is  possible  to  include 
models  of  exogenous  distrubances  by  augmenting  the  state 
space  X.  Then  some  linear  combination  of  the  states,  say  Dx, 
is  required  to  be  near  zero.  In  the  tracking  problem,  for 
example,  one  would  subtract  the  actual  state  from  the  exo- 
genous desired  control  state  to  get  an  error  in  the  state 
which  is  to  be  cancelled. 

Let,  then,  X denote  the  augmented  state  space,  and  let  D 
be  the  linear  observation  map  such  that  y = Dx  is  to  be 
regulated  near  zero.  If  D is  an  onto  map,  we  can  redefine 
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coordinates  of  the  state  space  so  the  D is  just  a projection, 
redefining  f and  all  other  maps  on  the  state  space  suitably. 

We  will  not  detail  this  procedure  here  but  refer  the  reader  to 
Wonham  [7] . 

Assume  then  that  by  proper  recoordinization  there  is  a 
subset  of  the  state  variables  which  are  to  be  stabilized  to 
zero.  Denote  this  subvector  of  x by  y.  It  would  be  sensible 
in  most  situations  to  define  a failure  zone  F which  y must 
not  enter,  and  assuming  y not  in  the  failure  zone,  we  might 
be  interested  in  the  maximum  average  error  in  y from  0.  This 
cost  corresponds  to  a 


R(x) 


y^F 

yeF 


We  won't  consider  cost  functions  which  include  the 
control  u or  the  comepnsator  state  z,  in  the  present  treat- 
ment. 


i 
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2.6  Defining  an  Ambiguous  Process  for  a Compensated  System 
In  Section  2.2  we  defined  ambiguous  measure  spaces,  and 
in  Section  2.3  to  2.5  we  defined  the  closed-loop  system 
associated  with  a finite-state  compensator  and  continuous 
plant.  For  the  rest  of  this  chapter  we  will  assume  that  the 
plant  and  compensator  are  fixed,  therefore  all  the  quantities 

defined  in  Sections  2.3  to  2.5  are  fixed,  such  as  all  the 

t 

quantities  in  Table  ( 1 ) , nv,no,  {(uxp)t>,  etc.  We  will  show 

. 

now  that  for  any  such  compensated  system,  am  ambiguous 
process  can  be  defined,  and  that  bounds  on  the  performance 
of  the  original  system  can  be  derived  from  bounds  on  the 
ambiguous  process. 

To  begin,  we  will  repeat  the  general  ideas  in  defining 

<2p  ambiguous  process,  although  somewhat  differently  then  the 

' 

discussion  in  Section  1.2  since  our  analysis  will  not  take 
place  in  quite  the  same  order. 

Suppose  that  the  compensated  system  is  running  and  we 
observe  its  progress,  watching  carefully  the  compensator 
state,  but  not  paying  too  much  attention  to  the  plant  state 
x(T).  Let's  say  we  observe  the  progress  of  the  plant  state 
only  within  some  finite  amount  of  detail,  so  at  observation  we 
distinguish  between  some  finite  number  of  conceptual  areas; 
for  example,  whether  the  velocity  is  positive  or  negative, 
whether  the  magnitude  of  the  position  exceeds  three,  etc. 

In  addition  we  might  remember  a few  past  observations.  Just 
knowing  the  present  conceptual  area  and  a few  past  observations 


will  not,  in  general,  enable  us  to  make  as  good  a prediction 
about  the  next  state  as  if  we  had  made  precise  measurements. 

On  the  other  hand,  the  limited  knowledge  we  do  have  will  still 
narrow  the  chances  of  the  various  possibilities  for  the  next 
state. 

Now  to  take  this  idea  one  step  farther,  suppose  we  were 
interested  only  in  predicting  the  next  area  of  the  plant 
state,  rather  than  the  exact  plant  state.  Available  to  make 
this  prediction  is  one  of  a finite  number  of  recollections  of 
the  area  of  the  plant  state,  and  knowledge  of  the  present 
compensator  state,  of  which  there  are  only  a finite  number  of 
possibilities.  By  a "history"  of  the  plant  state  we  mean 
information  up  to  and  including  the  present  area,  so  that  a 
history  of  plant  state  may  include  some  information  about  the 
present  area.  We  will  assume  that  the  information  about  the 
present  plant  state  area  is  sufficient  to  determine  which 
control  was  applied,  by  always  at  least  observing  which  coder 
equivalence  area  the  state  is  in. 

Given  this  finite  information  about  the  present  state 
(x  z) , there  must  be  some  range  of  probabilities  for  the 
probability  of  going  to  each  conceptual  area  in  the  next  step. 
Since  the  probabilities  might  depend  on  one  another,  we  can 
at  least  say  that  there  is  some  set  of  possible  transition 
matrices . These  possible  transition  matrices  will  be  the 
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histories  which  will  be  remembered.  Using  an  ambiguous 
process  model,  definite  bounds  on  the  performance  of  the 
compensator  can  be  calculated,  (and  quite  easily  because  of  the 
finiteness  of  the  model) , and  the  structure  of  ambiguous 
processes . 

Let  us  proceed  with  a more  formal  presentation.  We 
mentioned  that  at  each  observation,  at  least  the  coder 
equivalence  area  should  be  determined;  i.e.,  from  our 
observation  we  should  able  to  identify 


y = C (x) 


where  C is  the  coder  function,  otherwise  we  could  not  be  sure 
of  the  value  of  the  function  o(z,y),  i.e.,  the  control  applied. 
In  our  discussion  here  we  will  assume  for  simplicity  that  the 
conceptual  areas  of  X just  are  the  coder  equivalence  areas, 

C*1 (y) , yeY.  This  leads  us  to 

Definition.  An  immediate  area  of  the  f inite-> state  feedback 

T 

system  is  a vector  (y  z)  where  yeY,  zeZ. 


Definition . A history  it  is  a finite  string  of  immediate 
areas.  Letting  y^,y2»  • • *ymeY  and.-z^,  z2,  . . . zmeZ  , an  example 
of  a history  might  be  the  string 


*1.  *2,  , V 

’ = <z  ><*  I 

12  m 


For  any  history  we  require  that  the  following  property 
hold  between  the  elements  of  string  (as  shown  above) : 


T^i'Zi^  = zi+l  ^or  ^ = l»2,...,m-l. 


Definition.  A conceptualization  II  is  a finite  set  of 
histories  which  includes  every  immediate  area  (this  is  possible 
since  the  number  of  immediate  areas  is  finite) . 

To  introduce  the  concept  of  a journal,  consider  the 
following  example. 

Example.  Let  Y = {1,2},  Z = {1,2},  so  the  immediate 
conceptual  areas  are: 

Let  n = {(J;)  , (3)  » ' <1)  <1>  ' (25  (2}  ' (1)  (1}  (2}  } 

It  can  be  checked  that  n is  a conceptualization  for 
t:Y*  Z -*■  Z given  by  the  following  table: 


t (y, z) 


z = 1 

2 


y - 1 


1 

2 


2 

1 


Suppose  now  that  at  time  T,  tt  (T)  = (2)  (-,)  is  a history  of  the 
closed  loop  system.  Then  we  can  conclude  that: 


V-r 
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x(T)eC~ 

1 ( 2 ) 

1 

z (T)  = 1 

x(T-l) eC- 

(1) 

z(T-l)  = 2 

Clearly  z(T+l)  = 2,  so  the  next  immediate  area  must  be  (^) 

-1  2 

where  y = 1 or  2.  Suppose  y(T+l)eC  (2),  so  that  (2)  is  the 
observation  at  T+l . What  would  be  the  best  choice  for 
tt  (T+l)  ? 

2 

Clearly,  (^)  is  reasonable,  but  not  as  complete  a history 

12  2 12 
as  (^) (^) (2) • A history  like  (2) (2)  is  inconsistent  with  the 

12  2 

past  observations.  Since  (^) (2) (2)  is  the  longest  history  in 
II  consistent  with  past  observation,  we  choose  A as  the  "best" 
it  (T+l)  . 

Suppose  now  that  y(T+l)eC  '*'(1).  Then  the  longest  history 


K □ 


in  H consistent  with  past  observations  must  be  (2) 


The  key  observation  in  this  example  is  that  the  "best" 
choice  of  tt(T+1)  of  course  depends  on  y(T+l),  which  cannot  be 
known  with  certainty . Still , we  can  define  automaton  on  the 
history  set  II  which  receives  y(T)  as  input . 

Definition.  A journal  £ is  an  automaton  with  state  set  H, 
where  H is  a conceptualization,  and  input  set  Y (dependent  on 
II)  which  satisfies  the  following  conditions.  If 


yl  y2  ym 

tteII  and  tt  = (_x)  (/)  • • • ( m) 

Z1  Z2  2m 
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then  y ' , 

1.  (,?)=( 


m 1 


t ( zm , ym ) 
m ■*  m 


2 . m ' < m + 1 


y • » y 

3.  If  m'  > 1,  ( -j+m  m ■'")  = ( j)  for  all  je[m-m'+2,m] 

j+m'-m-l  zj 


4.  t maps  every  tt  to  the  longest  possible  string 
consistent  with  these  conditions,  f ! 


The  journal  x formalizes  our  notion  of  picking  a "best"  next 
history. 


Proposition.  x is  unique  for  every  n,Y. 

Proof . Suppose  II, Y are  fixed,  x^  and  x2  are  two  journals  on 
these  sets,  and  x ^ ( tt  , y ) = rr^ , x2(TT,y)  = tt  2 for  xxell , yeY. 
Condition  (3)  requires  that  if  the  length  of  tt.  is  the  same 
as  the  length  of  tt2,  then  tt^  = tt2.  Condition  (4)  thus  makes 
7Tt  = tt 2 r and  since  tt  and  y were  arbitrary,  x^  = x2. 

Suppose  now  that  the  journal  corresponding  to  some 
conceptualization  n were  hooked  up  to  the  feedback  system  as  an 
observer  of  the  closed  loop  system,  as  we  have  illustrated 
below.  Suppose  the  journal  were  started,  at  time  one,  rather 
than  zero,  in  the  state  (^jo))  • Then  the  reader  can  check 

from  the  definition  of  x that  the  subsequent  states  Tr(t)  of 
the  journal  all  represent  a correct  history  of  the  closed 


The  converse  is  not  true.  Suppose  n contains  the  two  histories 


tt,  = („  ) and  it.,  = („)(„) 


Z2  'Z1 


(and  possibly  others) . Given  that 


C (x (t-1) ) = y1  and  z(t-l)  = z ^ 

does  not  imply  that  -tt  ( t)  = t^,  for  it  may  also  be  true  that 

C (x ( t-2) ) = y2  and  z(t-2)  = z 2, 

in  which  case  tt  ( t)  could  not  be  tt,  but  must  be  tt2  or  some 
other  history,  since  tt2  is  a longer  string  consistent  with 
these  conditions.  Nevertheless,  by  requiring  that 


C(x(t-2)  ^ y2  and  z(t~2)  ^ z2, 

and  some  finite  number  of  additional  restriction  or  the  past 

coder  outputs  and  compensator  states,  we  must  be  able  to 

guarantee  that  TT(t)  = it..  This  set  of  events  will  be  denoted 

0 , or  0.  for  short. 

1 

Thus  we  can  write 


tt  (t)  = tt.  JJ,  (t) 
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: 


Now  consider  any  particular  Markov  Realization 

({\(T)},p  ) of  the  closed  loop  system  (see  Section  2.3). 

We  have  shown  that  { (z(tj ) > is  a Markov  process.  Since  each 

event  J3.(t)  depends  only  on  a finite  number  of  points  in 
1 

time,  we  are  led  to  the  following: 

Theorem.  For  any  Markov  Realization  {X(T)},y  ; { tt  ( t ) } is  a 
stochastic  process. 


Proof . We  will  define  this  stochastic  process  by 

a)  Defining  the  state  space.  It  is  just  n.  (7r(t)  is  not  a 
state  in  the  systems  sense,  but  iT(t)  is  the  state  of  a 
stochastic  process.) 

b)  Taking  Index  Parameter  t to  be  in  Z+ . 

c)  Defining,  for  a finite  subset  of  indices  t^tjf.-.t  , the 
joint  distribution  of  tt  (t^)  , . . . tt  ( t£)  to  be 


Pr^1T^tl)  = Kl' »"(tn)  = = Pr ^1  * fcl^  ' * * * * fcn^ 


which  is  well  defined  because  (x  z) ’ is  a Markov  process, 
the  sets  of  condition  depend  on  only  a finite  number  of 
time  points,  and  each  of  the  sets  C ‘'"(y),  yeY,  is  a Borel 
set  by  assumption,  and  this  measurable  by  (u x p)T 1 


We  now  give  our  first  definition  of  the  ambiguous  process 
associated  with  n. 
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Definition.  <3^(11)  is  the  set  of  all  stochastic  processes 
{n (t) } , {X (T) } nv#  nQ  . 

Now  *1  is  one  of  those  objects  which  could  only  in  the 
most  rare  cases  actually  be  calculated,  and  whose  mere 
existence  can,  by  many,  be  restricted  to  the  domain  of  a 
lively  mathematician's  imagination.  Our  only  ambition  in 
making  such  an  abstruse  definition  is  to  pinpoint  exactly  what 
it  is  we  would  ideally  want  the  ambiguous  process  to  be.  For 
all  practical  purposes,  the  set  must  be  padded  out 
sufficiently  that  it  admits  some  kind  of  analytical  descrip- 
tion, which  can  then  be  used  to  calculate  the  performance 
bounds.  That  is  our  next  task. 

First,  let  the  number  of  elements  in  n be  N.  Let 

be  closed,  convex  subsets  of  rN  such  that  for  any 
pe^,  u p = 1 and  p ^ 0 component  arises,  where  u = (1  1...1)  . 
We  will  eventually  interpret  the  p's  as  probability  vectors  of 
going  from  one  state  to  each  of  the  N other  states. 

Consider  the  set  of  all  finite  length  strings  consisting 
of  elements  of  II;  denote  it  by  II*.  will  always  denote  an 

element  of  H*  of  length  T. 

Let  ie[l,N]  and  o^ell*.  Let  p^(wT)  be  an  element  of  P^. 

Now  consider  a set  of  choices  {p^fto.p)}  for  all  i = 1,2,... N, 

Define  a probability  for  each  string  in  JI*  as  follows: 

( 1 if  n .=  n (0) 

1.  Pr(TTi}  =<  1 

( 0 otherwise 
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2.  For  any  a^ell*  of  length  T _>  1,  let  be  the  last 
element  of  u>T.  Then  define: 

= Pr{wT}  • Pi(o)T) 

where  Pi(wT)  A is  an  N-vector . □ 

It  is  clear  that  the  choice  of  the  p^Cu^),  i = 1,...,N, 
tonsil*,  determine  the  probability  of  every  string  in  n* 
uniquely.  Furthermore,  there  is  an  obvious  stochastic  process 
for  each  set  (p^(u)T)};  the  axioms  of  probability  can  be 
checked,  and  a measure  on  every  finite  length  string 
determines  all  the  joint  probability  distributions.  We  denote 
the  stochastic  process  by  ^(p^(ojt)}. 

We  are  now  ready  to  close  the  loop. 

Definition.  The  ambiguous  process  on  where 

is  a closed,  convex  subset  of  the  probability  vectors  in  rN, 
is  the  set  of  all  ^{p^  (u>T)  } , for  all  possible  sets 
(pi(wT)},  in  which  pi(aiT)e/?,  i = 1,...,N  Vw-eJI* . 

We  shall  often  denote  , . . . P..)  by  P. 

1 N u 

to  find  a P such  that 


Pr^a)T7rl^ 


VVV 


Our  goal  is  now 
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Definition.  A transition  bound  is  a mapping  (subscripted  by 


*3-  : N 

w u x 


such  that 


X (nx)  {y’-u  = Fu(y ' , a (t) ) ,y ' e nx/A(T)env) 


Remark.  Transition  bound  always  exists,  since  RxeN  . (In 
Chapter  3 we  will  see  that  the  right  hand  side  is  usually  in 

no 

N anyway,  so  knew  bounding  is  necessary.)  Let  S.,...,S  be 
x 1 m 

N 

Borel  sets  in  R . 


Definition. 


S ^ . 


I y(S1)\ 

(n)  = { y(S2}  , yen } 


y(Sm> 


Y1  y2  ym 

Definition.  Let  tt  = (_  )(  )•••(  ) en  . 

Z.  Z-  z 

12  m 


\ PC-1  (m)^a  (y^,  V-l*  PC"1  (ym.i)  * * *^o  (yr  z±)  PC"1  (yx) 


pn-l  fv. ) 


tt  -c|[cvxhull[EVl(l),...C-l(r)  ($?(y  ,z  )(\))1] 


r N 

where  E:R  -*■  R ; if  E(v)  (i)  is  the  ith  component  of  E(v), 


v a vector  in  Rr, 
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E(v) (i)  = 


if  VyeY  Viren,  T(iT,y)  ^ it. 
y-j  where  y^  is  the  unique  element  of  Y 


such  that  x(ir,yj)  = tt^ 


and  P is  the  projection  operation  defined  in  2.2 


□ 


These  notational  monstrosities  actually  turn  out  to  be 
useful.  We  now  complete  this  development  of  a simpler  form 
for  ^ (IT)  in  stating  the  following 

Theorem.  d(fi  ) 0^(11). 

*1  UN  1 

Proof . The  reader  should  recognize,  if  he  has  studied  the 
definitions  carefully,  that  this  theorem  is  really  a 
notational  paper  dragon.  We  will  go  through  a detailed  proof 
to  make  the  concepts  more  clear. 

Let  &1(H)  be  a stochastic  process.  By  the  definition 
of  there  must  be  some  { X ( T ) } nv,  nQ  such  that 

{ 7t  ( t ) } is  the  stochastic  process  associated  with  the  journal 
x of  the  closed-loop  system.  All  we  need  to  do  to  show 
SzOifi)  is  to  find  some  choice  set  {p^(wT)}  which  is  consistent 
with  P and  generates  Consider  any  w_ell*  and  i = 1,2,...N. 


Let  us  calculate  p^(u)T)  for  the  stochastic  process  & , so  that 
we  can  show  that  p^(wT)e/?,  and  then  (pi(ioT)}  is  consistent 

with  P. 

Now 


f 


Expand  the  it's  graphically: 
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Case  1 . oj,p  inconsistent.  If  the  y's  and  z's  in  some  column 
don’t  agree,  wT  is  inconsistent,  i.e.,  Pr{^T>  = 0.  Obviously, 
?i  (^t ) no^  ine^  in  this  case,  so  we  can  choose  it 

to  be  if  we  wish. 


Case  2.  ooT  consistent.  Here  obviously  pr{wT}  > 0,  so  again  if 
Pr^wT^  = we  coui^  still  take  p^(cuT)e^?.  The  only  remaining 
possibility  is  pr{wT)  > 0 and  coT  consistent,  which  is  what 
we’re  after. 

Consider  all  the  outcomes  of  { ( )}  which  would  have 

z (t) 

caused  co^.  We  know  this  is  nonempty  since  pr{w,p}  > 0.  tt^  is 
the  last  element  of  the  string  tuT,  which  is  too  say  7T(t)  - tt  . . 
Let 


7T  . = ( ■ 

1 z. 


y 

(,m) 

zm 


• v (f  ) 

Consider,  for  each  outcome  in  ((Jr!)} 

z V t ) 

ojT,  the  probability  measure  of  x(T-m- 
Knowing  that 


which  would  have  caused 
1)  , call  it  UT_m_j_. 


Cx(T-l)  = ym,  z(T-l)  = zm 
• • 

• • 

Cx(T-m)  = yJL,  z(T-m)  = z 


and  since  (x  z)  is  a Markov  process,  the  measure  of  x(T-m) 
must  be 

PC-1(y1)Fa(y1z1)uT-m-l  ' 


i 
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Thus  we  have 


“T-l  - PC-l(ym)Fo(ym.lZm.1)---pc-l(y1)Fa(y:lz1)>'T-m-l 
Let  n be  the  set  of  all  possible  then  by  the  definition 

of  T. 


nu:5n  (PC-l(m)  o(ym_lZ]n_1)  •••pc-l(y1)Fo(y;lZl) 


yT-m-l* 


since  Fa (y^z.^ yT-m-l  must  be  in  5 • The  probability  of  getting 

each  of  the  r coder  outputs  at  time  T,  given  some  uT  ^erii  is 
just 

Vl(l)  . . .C-l  (r)  Fo  (y  z ) (yT-l} 

mm 

so  these  probability  vectors  must  all  lie  in  the  set 

*C-1(1)  , . . .C-l(r)  {%(yz  ) (V  } • 

m m 

We  are  almost  there,  as  we  now  only  need  to  check  that  the 


various  probabilities  for  going  to  1^,...^  are  in  tfP.  The 
function  E is  designed  to  "expand”  Tc_1(1)  c~l(r)  just  so 
this  will  be  true.  □ 

Example . We  return  to  the  example  given  in  the  Introduction, 
only  formulated  probabilistically  so  we  can  show  how  the 
ambiguous  measure  sets  arise  naturally  in  the  analysis. 

We  take  X = U = v = R,  Z = { z ^ , Y = {yQ , yx , y2 , y3 } , 


tr. 


f(x,u,v)  = 3/2 x + u + v 


t (y, z)  = z 


a (y, z) 


if  y 
if  y 
if  y 
if  y 


*1 

^2 

^3 

yo 


C (x) 


if  xe(-°°,-2)  or  (2,<=°) 
if  xe [-2,-1) 
if  xe [-1, 1] 
if  xe(l,2] 


Let  Nx  and  Nv  be  as  defined  in  Example  1,  Section 
n = {X  } where  X is  the  uniform  measure  on  [-1/2 

'V  V V 

n = (u  } where  u is  the  uniform  measure  on  [-1,1] 
o o o 

there  is  only  one  possible  Markov  realization,  so 
a Markov  process. 


.2,  with 
1/2] , and 

Thus 

,T  . 

X z)  IS 


The  performance  criterion  for  this  system  is: 


(x) 


x2  if  C (x)  f Yc 
°°  otherwise 


To  illustrate  how  ambiguous  process  OLOfo  can  be  defined  for 
this  feedback-system,  define  a conceptualization: 


y0  y2  y3  yi  yi  y3  y3 

n = {(  °)f( /),(  2),  (z  ),(z  ) (z  )'(z  ) (z  )> 

zi  zi  zi  zi  zi  zi  zi  zi 


and  a transition  bound: 


: N -*•  N . 
V U X X 


Recall  that  ^ is  a transition  bound  if  for  every  neN  , 

12  XX 


^(nx)  3(u:y  = Fu(y  ' ,X  (T)  ) ,\i'  enx/X  (T)  env} 


We  digress  momentarily  to  discuss  F (y',X),  which  is 
defined  by  f.  Let  (2/3y‘)  be  the  measure  defined  by: 


( 2/3y ' ) (B)  = y * (2/3B)  (BeB) 


If  the  measure  of  x(T)  is  y 1 , then  the  measure  of  3/2x(T)  is 
(3/2y'),  and  the  measure  of  3/2x(T)  + v(T)  is  (2/3y')  * X so 


Fu(y',X)(B)  = [ ( 2/3y ' ) * X] (B-u)  (BeB) 


We  will  not  use  this  formula  for  explicitly  because  we 
will  be  able  to  derive  the  result  intuitively  for  the  simple 
y ' , X in  question. 

For  any  probability  measure  y there  is  some  set  S such 
that  y(S)  = 1.  The  support  of  y is  the  inf imal  S with  this 
property.  If  nx  is  a set  of  measures,  the  inf imal  support 
of  nx»  denoted  by  infsp(nx)>  is  the  infetmal  S such  that 
y (S)  = 1 for  all  yenx.  We  now  take  as  a transition  bound: 

u(nx)  " ?infsp(Fu(nx,Xv) ) 

which  clearly  satisfies  the  necessary  properties. 

These  two  items,  the  conceptualization  II  and  the 
transition  bound  are  all  that  is  needed  to  define  the 
ambiguous  process  ( IP ) . We  start  by  computing  the  for 

y-. 

ire II . Let  it  = (z  )»  for  example.  Then 

nTi  “ Pc~i  (y^ ) ^ = C(-2,-l]  • 

It  is  more  difficult  to  derive  a for  a history  such 

y y 

as  tt  = ( 1)  I1)  , for 
1 Z1 

n tt  " PC-l(y1)  o (yx,  PC-1  (y.^  ^ 


The  first  step  is  to  compute 


^a(y1z1)PC-l(y1)C  ^+2C[-1,-1] 


If  xe [-2,-1] , then  3/2x  + ue [-1,1/2)  and  3/2x  + u + ve  [-3/2,1) 
We  conclude  that  inf sp (F2 ( [-2 , -1) , \ ) ) = [-3/2,1),  so 


+2^  [-2,-1)  ~ C(-3/2, 


1) 


Finally,  PC-1 (Yl) 5 (-3/2 , 1]  P [-2 , -1) 5 [-3/2 , 1]  ~ C[-3/2, 


1) 


Theri  , irell,  are  summarized  in  the  table  below. 


We  can  now  compute  £r\,  i = 1,...,6,  where  the  histories 
are  indexed  as  shown  in  the  preceding  table.  Let  us  take  as 
an  example  the  computation  of  P : 
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^1  E<PC-1(1)  C-l  (4)  9?(ziyi)  (C[-2,-l)^ 

First  we  tackle  the  problem 

^C-l(l)  . . .C-1(4)^C25  [-2,-1) 

= * [-2,-1)  , [-1,1]  , (1,2]  ,(-<*>, -2)  U (2,«)  9^  [-2,-1) 

We  will  find  the  solution  intuitively.  Suppose  x (T) e [-2,-1) ; 
then 

x (T+l) e3/2[-2,-1)  + 2 + v (T) 
e [-1,1/2]  + v (T) 

Since  |v(T)|  <_  1/2,  the  probability  that  x (T+l)  e [-oo, -2)  , is 
zero,  i. e. , 

¥[-~,-2)  [ °fl ^ [-2,-1)  ] = 0; 

and  also 


9 


(-2,oo)  [925  [-2,-1)  ] ” *(1,2]  [ 5*2 5 [-2,-1)  1 = 0 ’ 


The  probability  that  x (T+l) e [-2, -1)  is  between  0 and  1/2, 
depending  on  the  exact  probability  measure  for  x(T).  Thus 


?C-1(1)  . . .C-l(4)  ^25  [-2,-1)  " 
Finally,  P±  = cl  [cvxhull [E { ^ jaj  : 


ae  [0,1/2] 


ae [0,1/2] }] ] 
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Yi 

To  see  3ust  what  E does,  suppose  that  for  some  T,  tt(T)  = ( ) 

Z1 

From  the  above  computation  we  know  that  the  probability  of 
x(T+l)eC  1(y1)  is  a,  where  a is  between  0 and  1/2.  Hence 
there  is  a probability  between  0 and  1/2  that 
yl  yl 

tt(T+1)  = tt_  = ( ) ( ) . Similarly  there  is  a probability 

o z ^ 

between  1/2  and  1 that  x (T+l) eC-1 (y2) , so 

y 2 

tt(T+1)  = tt_  = ( ) with  probability  between  1/2  and  1, 

z Z1 

is  no  probability  that  tt(T+1)  is  t^,  tt3  , tt4 , 7Tg  . Thus 


: ae ( 0, 1/2] > = 


0 

1-a 

0 

0 

a 

V 0 


: ae ( 0, 1/2] 


Since  this  set  is  closed  and  convex  it  is  just  (P^. 
can  be  computed  similarly  and  are: 


There 


t P-  (/>.. 


r 0 

[0,1] 

0 

[0,1] 

0 

0 

U/2,1) 

[0,1] 

[1/2,1] 

[0,1] 

1 

1 

0 

[0,1] 

0 

[0,1] 

0 

0 

0 

0 

0 

[0,1] 

0 

0 

[0,1/2] 

0 

0 

[0,1] 

0 

0 

0 

0 

[0,1/2] 

[0,1] 

0 

0 

where  we  have  simplified  the  notation  (no  information  has 

been  lost) . Another  convenient  representation  is  shown  in 

y ? y 3 

Figure  (3.1).  Here  (33)  stands  for  it,,  i.e.,  ( )(  ). 

o z 2 z 2 


Figure  2.1 

We  can  now  immediately  see  that  since  tt(0)  = tt2, 
x (T)  will  never  exceed  2_  since  all  supports  of  the  above 
histories  lie  in  [-2,2]. 

We  can  also  calculate  max imum  square  deviation  for 
each  tt e II ; they  are 


History 

Maximum  square  error 

0 

00 

1 

4 

2 

1 

3 

4 

11 

9/4 

33 

9/4  Ta) 

and  bound  to  the  mean  square  error  we  rely  on  a theorem  to  be 
proved:  that  some  choice  of  the  transition  probabilities 

(in  Figure  2.1) , all  at  their  extreme  points , will  give  a 


lower  bound  on  the  mean  square  error . This  lower  bounds  even 


6 


the  nonstationary  possibilities . Using  this  theorem  it  is 
easy  to  see  that  a lower  bound  will  occur  by  choosing  the 
probabilities  as  shown  in  Figure  2.2: 


The  steady-state  probabilities  are: 


History 


S.S.  Prob 


so  the  maximum  square  error  is  2/5(1)  + 2/5(4)  + l/5(9/4) 
2.45  and  thus  the  worst  root  square  mean  error  is  1.57.  f 


68 


2 . 7 Bounds  on  Performance 

In  this  brief  section  we  will  show  how  the 
performance  of  a compensated  system  can  be  related 
to  the  performance  of  an  associated  ambiguous 
process . 

Definition.  Let  be  a stochastic  process. 

Define,  in  parallel  to  our  earlier  definitions, 

T v 

J ( £ ) = inf  ( a : Pr  £ lim  sup  ^ SZ  r ( ^ (T)  ) * a ) 
( l T-%*.  t=0 

. “I 

J(Q)  = sup  ^ b : b=J(^),  $ $ 
where  r(<7ri)  = sup  ^ R(x) : C(x)=yi|  , 

IT  . = (^1)  • • • (^m) 


*■  __ 

Proposition . J ^ J for  ™ an  ambiguous 
model  of  the  closed  loop  system. 


process 


Proof : Suppose  f'/UT)^  , yUg  is  a Markov 


realization . 


If  lim  sup  — 21 

T-*oo  t=0 


r ( 'J'(t) ) < a then 


lim  sup  ^ R (x  ( t ) ) < 


T —s 


2.8  Ambiguous  Processes 


In  the  previous  sections  we  have  seen  that  any 
finite-state  control  system  can  be  modeled  as  an  ambiguous 
process  C£(<P)  , given  a conceptualization  II  and  transition 
bound  ^ . Of  course  the  computation  of  <P  directly  from  the 
formulas  given  there  are  still  out  of  the  practical  realm, 
but  we  postpone  the  discussion  of  the  practicalities  of  this 
compution  to  Chapter  3 so  that  the  general  theoretical 
development  can  be  continued.  Let  us  begin  with  a brief 
review  of  ambiguous  processes  as  defined  haphazardly  in 
previous  sections. 

Defn.  p e (RN  is  a probability  vector  if  ( l***l)p  = 1, 
and  p ^ 0 component-wise. 

Defn . An  ambiguous  process  ^ on  a finite  state  set 

X is  an  N-tuple  of  closed,  convex  subsets  of  probability 

vectors  in  <RN.  We  will  denote  these  subsets  by  (Pi,  . ..,  (P  , 

d = ( Jj.  . .(P^)  > and  <P  the  set  of  all  matrices  with  the  ith 

column  from  (P . . 

i 

(We  choose  X here  for  the  state  set  to  avoid  having  so  many 
"tt"s  in  the  text.) 


Defn . The  stochastic  processes  associated  with  (P  are 
all  those  that  assign  probabilities  to  elements  of  X*,  the 
strings  of  finite  length  from  set  X,  such  that: 


where  5 is  the  "cost"  function  for  a Markov  process,  defined 
in  terms  of  limiting  state  probabilities.  We  will  go  on  to 
further  characterize  P*  by  saying  that  for  some  vector  h, 
each  column  of  the  row  vector 


T 

h P* 


is  maximum  over  all  p e <P.  For  certain  cases  of  <P  the 
condition  becomes  even  simpler.  These  facts  make  ambiguous 
processes  genuinely  useful  for  determining  bounds,  and  this 
will  be  fully  explored  in  Chapter  3.  For  now  let  us  outline 


%*\ 
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the  progression  in  the  remainder  of  Chapter  2.  First,  we  will 
review  the  standard  results  on  finite-state  Markov  Processes, 
then  take  up  differential  properties  of  limiting  state 
probabilities,  and  define  the  function  5.  The  first  result 
will  be  that  z,  has  a maximum  in  <P . This  maximum  will  then  be 
shown  to  bound  J,  and  necessary  conditions  on  P*  will  be 
derived. 

2.9  Markov  Processes 

Suppose  <P  - P consists  of  a single  matrix.  Then  CR(. P) 

is  a Markov  process;  to  see  this,  let  ojt  be  an  outcome  to  time 

T,  (x.  , x.  , . . . . x.  ) , and  for  any  x.  , 

■ xt  XT+1 


So  in  this  section  we  will  take  up  a "special  case"  of 
ambiguous  process,  namely  a Markov  process.  We  denote  a 
Markov  process  by  its  N by  N transition  matrix  P.  We  will 
differ  from  the  standard  arrangement  of  probabilities  in  a 
stochastic  matrix  by  requiring  that  the  columns,  rather  than 


& 
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l 


m. 


the  rows,  sum  to  one,  but  of  course  require  that  every 
element  of  P be  nonnegative.  We  are  now  consistent  with  the 
format  of  matrices  in  the  set  <P  associated  with  any  ambiguous 
process . 

A standard  result  from  the  theory  of  Markov  process  P 
can  be  partitioned  in  to  classes  of  the  following  kind  (see, 
for  example,  Gantmacher  [13] , Vol  II) : 

1)  Transient 

2)  Ergodic 

3)  Cyclic 

so  that  the  probability  of  making  a transition  from  one 
non- transient  class  to  any  other  class  is  zero,  and  each 
kind  of  class  has  certain  characteristic  properties. 

The  probability  of  the  state  being  in  a transient  state 
more  than  a finite  number  of  times  is  zero.  This  is  the 
characterzation  of  transient  states. 

For  Ergodic  classes,  states  reoccur  infinitely  often, 
and  furthermore,  the  probability  of  being  in  any  state 
becomes  independent  of  the  initial  state,  and  grows  toward  a, 
constant  as  T gets  large. 

For  Cyclic  classes,  states  reoccur  infinitely,  but  unlike 
ergodic  classes,  the  progression  of  the  states  with  time 
follows  a strict  circular  sequence. 
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Now  suppose  that  Xq,  the  starting  state,  is  deterministic. 
Only  certain  of  the  transient,  ergodic  and  cyclic  classes  are 
reachable  with  non-zero  probability  from  x^ , and  in  this  ana- 
lysis we  will  only  be  interested  in  the  ergodic  or  cyclic 
classes  reachable  from  xQ,  which  will  be  denoted  0XQ  = fDi^' 
or  lD  for  short,  where  each  is  an  ergodic  or  cyclic  class. 

Defn.  A set  of  states  is  unified  if  it  forms  an  ergodic 
or  cyclic  class.  If  the  entire  set  of  states  of  P is  unified, 
then  we  way  that  P is  unified. 

The  transient  and  unified  classes  of  a general  stochastic 
matrix  can  be  represented  by  an  illustration,  or  by  a block 
diagonal  matrix  as  shown  below.  We  will  always  restrict  our 
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If  e (Q  is  a unified  class,  let  PQ  be  the  block  of  the 

1 

matrix  P corresponding  to  the  intra-transition  probabilities 
for  the  states  in  . Clearly,  by  the  block  diagonal  structure 
of  P (excluding  transient  states) , PD  will  again  be  a stochatic 

i 

matrix. 

A property  of  ergodic  classes  is  that  there  must  be  a 
vector  of  limiting  state  probabilities,  which  obeys 

Pq  Poo  = Poo  , U Poo  = 1 , Poo  0 (U  = ( 1 * * * ) ) 

i 

where  is  the  ergodic  class.  This  leads  us  to  a natural 
definition  of  the  average  cost  of  being  in  , 

N 

c0O(Di)  = l c(x.)  Poo(j) 
j=l  3 • 

For  cyclic  classes  it  is  equally  easy  to  define  an  average 
cost.  Suppose  we  took  it  to  be 

i T 

lim  i l c ( x ( t ) ) 

T + OO  t = l 

Since,  given  a starting  state  xQ  € dJ  in  cyclic  class  D|,  the  x(t) 

must  follow  a prescribed  circle  x.,x0,...,x  , the  above  limit  must 

x ^ m 


converge  to 


We  are  interested  here  in  the  behaviour  of  c,  when  small  changes 

are  made  in  P.  To  this  end  we  define  a feasible  direction  A to  be 

an  N by  N matrix  such  that  for  small  enough  e>0,  P + eA  is  a 

stochastic  matrix.  Of  course,  for  A to  be  a feasible  direction, 

T 

u A must  be  zero.  If  P - eA  is  a stochastic  matrix  as  well,  we 
say  that  A is  a bifeasible  direction. 


we  will  always  mean 

d £ (P+eA  ) 
de 

and  this  applies  as  well  to  any  other  functions  of  P,  such  as  p , 


Theorem.  Let  P be  ergodic  and  let  A be  a bifeasible  direction. 


exists , and 


= Mp 


Proof:  Since  P is  ergodic,  there  must  be  a p with  Pp  = p Since 
■ - ■ 00  00  00 

A is  bifeasible,  no  element  P(i,j)  can  be  zero  in  P if  A(i,j)^  0, 

thus  for  small  enough  e , P±eA  will  be  non-zero  everywhere  P is  non- 
zero, so  P±sA  will  be  ergodic  and 
(P+eAJp^e)  = p^e) 

Rewrite  these  equations  as 


(I-P)  (* 


“)  = AP 


Let  us  turn  to  the  infinite  sum  M. 

Let  v be  an  eigenvector  of  P,  Pv  = Xv, Ixl  =1.  If  X=0,  then  Pv=0, 

T T T it  ^ 

and  since  uP  = u,uv=0.  On  the  other  hand,  if  0 <|Xl^l,  then 

T T T 

u Pv  = u v = u Xv 

so  again  uTv  = 0.  (uT=(l  1 ...  1)).  Thus,  if  |x|<l,  then  uTv  = 0.  But 
for  an  ergodic  matrix  P,  the  proposition  guarantees  that  all|x(<  1 
except  for  one  eigenvalue  corresponding  to  pm  . Let  A^  be  any  column 
of  A , and  suppose  that  P has  a full  set  of  eigenvectors.  Then  A^ 
could  not  have  any  component  in  the  direction  of  p^;  if 

4i  ' C1V1  * •••  * V* 

then  taking  v^=  pM, 


0 
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0 T.  T . . T 

= u A.  = C]u  Pao  + ...  + cNu  vN  = c1 

so  c^=0.  For  the  case  of  a full  set  of  eigenvectors,  then  M must 
exist  because  every  eigenvector  component  in  has  magnitude  less 
than  one.  This  argument  can  be  extended  to  P not  having  full  set  by 
using  extended  eigenvectors. 

Since  Oa 


m = y pnz 


n=0 

M must  satisfy  the  equation: 


and  thus: 
Since 


(I-P)M  = A 

(I-P)MPoo(e)  = APoo(e) 

(I-P)  (Stt(e>  - P°°)  = Ap  .(e). 


and  by  the  additional  constraints 

UT  (£°^(.e?  ^ Eg)  = o 
€ 

uTMPoo(e)  = 0 

we  must  have  (since  the  rank  of  I-P  is  N-l) : 


P (e)-p 

c co  00 


= Mp  (e) 


Certainly 


lim  Mp  (e)  = Mp  = lim 

00  oo 

e->  0 e->0 


Poo' -Poo 


dPo 


dA 


Similiarly 


so 


M(e)  - M 


M2  = * 

4 A 


= MM(£) 


□ 


Now  let  us  turn  to  the  properties  of  £.  For  an  ergodic  P,  £(P)  = 
cTp  . Suppose  that  (^is  a closed,  convex  set  of  ergodic  P,  in  which 
the  columns  are  independent  (c.f.  section  2.8.) 


Theorem.  For  the  **  described  above,  £ achieves  a maximum  at  an 
extreme  point  in  P . Furthermore,  for  some  h,  hTP*  hTP 
(componentwise)  for  all  Pzfi. 

Proof ; Since  every  P <£(P  is  ergodic,  and  P is  closed,  then  5 
is  differentiable  on  P,  hence  (Avriel  [15] ) 5 achieves  a maximum 

somewhere  in  P,  call  it  P*.  At  P*  we  must  have 


^ < 0 

dA  ^ 

for  all  feasible  A . Write  A = ( A^  A2  ...  A^)  and  let 


A 


0 -1  -1  ...  -1 

0 1 0 ...  0 

0 0 1 . . . 0 

Vo  0 0 ...  i 


, hT  = cTM(A).  Then  cTM(Ai)  = (0  . . 


. hTAj 


.0) 


and  cTM(A)  = hTA  . Since 


dC  T T ^ 

TT~  = C M(A)p  = h p <0 
d A 00  00  ^ 

T 

we  conclude  that  h A ^0  (since  >0)  and  thus 
hT  (P*+A)  = hTP*  + hTA  hTP* 


Furthermore  this  condition  can  always  be  achieved  at  an  extreme  point. 
Corollary.  Suppose  the  simplexes  P^  are  defined  by 

'pu\ 


IN/ 


! Pil€^1l'1i^ ' pi2*^12'12^  " * ' ' PiN<^1N'1N^ 

N 


l 

j=l 


Pij  =1 


(P 


where  all  0^  lj  <1.  Then  w satisfies  the  conditions  of  the  previous 


theorem,  and  P depends  only  on  the  ranking  of  the  elements  in  the 


vector  h. 
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Proof:  For  the  simplex  in  RN  defined  by  each  (P. , the  solution  to 


T * T 

h P^  ^ h P^  will  be  achieved  by  putting  the  greatest  amount  of 


probability  mass  in  the  component  with  highest  h multiplier,  and 
* 


so  forth.  Thus  P depends  only  on  the  ranking  of  the  components 
of  j from  the  least  to  greatest.  1 1 


2.11.  Maximization  of £ in 


P 


We  now  take  up  the  general  question  of  the  existence  of  a maximal 
£(P*)  in  an  arbitrary  P satisfying  the  conditions  for  an  ambiguous 
process.  Elegant  results  are  obtained  by  restricting  P to  be 


ergodic,  and  were  derived  in  Section  2.10.  The  same  results  are 


true  for  the  general  case,  namely,  that  P is  an  extreme  point, 


and  that  for  some  h,  nP  is  minimized  componentwise,  but  unfortunately 


the  proofs  are  very  tedious.  We  will  outline  here  the  proof  that 


p exists. 


Theorem. 


attains  a maximum  in 


in  P. 


(We  show  the  explicit 


dependence  of  ? on  Xq  because,  for  general  P,  £ is  not  well-defined  if 

a starting  state  is  not  specified. ) 

Outline  of  Proof : We  will  only  sketch  out  the  major  ideas.  Assume 

to  the  contrary  that  no  maximum  exists.  Then  there  must  be  an 

ascending  sequence  (C(P^),  5(P2),  •••  ) with  no  upper  bound  C(P*) 

in  (P.  We  will  derive  a contradiction. 

Let  the  sequence  {Pi } be  such  that  the  5 of  each  P . is 

1 x0  1 

the  same;  this  is  possible  since  the  number  of  states,  and  hence  the 


unber  of  partitions,  is  finite.  Furthermore,  let  {p^.}  be  a sub- 


sequence  which  converges,  say  to  P.  Since  IT  is  compact,  p € (r 
The  partition  of  each  Pk  is  the  same,  but  the  partition  of  P may 
be  different.  Let  us  consider  the  possibilities. 

If  x1  and  x2  were  in  different  classes  in  <D,  the  partition  of 
each  Pk'  must  be  different  classes  in  £)  , the  partition  of 

P,  since  lim  P(x1,x2)  = 0.  Any  cyclic  class  in  tD  must  also  be 

k-*«» 

cyclic  in  <D  although  the  class  may  not  be  in  <0  if  they  are 
not  reachable  from  x^. 

Of  the  transient  states  in  (D , some  may  break  off  into  unified 
classes  in  € , or  become  unreachable  entirely.  Similiarly, 
ergodic  classes  in  0 may  fragment  into  transient,  ergodic,  and 
cyclic  subclasses.  This  is  illustrated  below. 


*• 

ttttirciekt’ 

«r^»4ie 

Cvjtlic 

Cyclic 

U*rceck«Ut 


iun>{ic4 


Cuol't-  c^cJic. 


here  for  the  special  case  (P  strictly  positive,  i.e.,  all  P « (p 
strictly  positive.  Thus  all  PtfPare  ergodic.  The  general  proof, 
accounting  J;or  the  discontinuities  of  can  be  constructed  along 


the  general  lines  outlined  in  the  previous  theorem  on  the  exis- 
tence of  P*. 

Also,  in  proving  this  theorem,  we  will  make  one  assumption  on 

the  behaviour  of  outcomes  of  the  ambiguous  process.  Let  ^ be  a 

stochastic  process  in  @ , let  ft  be  the  set  of  all  outcomes,  and 

let  weft.  Denote  the  outcome  oj  up  to  time  T be  u^,  and  let  x^eX.  Then 

there  must  be  some  set  of  times,  call  them  {T^}  » for  which  x(T)  = 

x^  in  the  outcome  to.  This  set  may  be  empty,  and  it  may  be  all  TeZ+. 

Now  at  each  of  these  times,  T^,  the  probability  that  x(T^  + i)  will 

be  each  of  x^,x2,...,xN  X given  wT  is  some  vector  p^(u^,);  thus 

the  probability  that  x(T  +1)  is  some  particular  x.  given  w is 

K 3 T 

p.  (u>  ) (j) , or  P (i,j)  for  short.  Now  for  this  particular  outcome 
to,  and  for  each  T , either  x(T  +1)  = x.  or  x(T,  +1)  ^ x..  Let 

K Jc  J ^ D 

Zj (Tk+1)  be  the  indicator  of  the  former,  so  that  z^ (T^+l)  = 1 iff 
x(T,+l)  = x..  For  outcomes  with  {T  } infinite,  we  would  expect  that 


Urn  / 

n~  k=l 


since  the  trials  of  Zj=l  have  been  in  some  sense  "independent",  and  all 
have  uniformly  bounded  variance.  If  ui  has  this  property,  we  call  it 
well-tempered.  In  our  proof,  we  restrict  our  attention  to  well- 
tempered  outcomes  . Proving  that  the  well-tempered  outcomes  have 


probability  one  is  beyond  the  scope  of  this  research. 
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Before  turning  to  the  proof  of  the  bounding  theorem,  we  will 
show  by  counterexamples  that  each  property  of  the  set  (P  plays  a 
non- trivial  role. 

Counterexample  1.  Non-closed  ambiguous  process.  It  is 
clear  that  P*  may  not  exist  if  (P  is  not  closed.  In  this  counter- 
example we  show  that  P*  can  exist,  but  ?(P*)  is  still  not  a lower 


bound  on  expected  return. 

Let  tP1  - { (*)  : x [j,l),  y=l-x  } 

<P2  - { (J)  } 


u»>0 


c (ir  ) = 0,  c (tr  ) = 1. 

1 2 

Clearly  the  limiting  partition,  for  all  Pe  , 
is  { { -ir2 } } , and  c^n^}  = 1.  Thus  take  as  P any  element  of  P so 
CtP*)  = 1. 

Now  consider  the  following  realization. 


P(T)  = 


,1-  e 


-T 

:(-2  L) 

-T 

(-2  ) 


, T=0 , 1 , . . . 


The  probability  that  this  Markov  process,  when  started  in  state  tt^. 


will  stay  in  ir^  forever  is 


CO  _IT>  I 2 

n e<“2  > = e^0 

T=0 


-T 


= e 


> 0 


-2 


so  there  is  a probability  of  e that 

T 

I 

T-*»  * t=l 


1 T 

1 = C(P*)  ^ lim  inf  — £ c(x(t))  = 0,  so  (P*)  is  not  a lower 


bound. 


L 


2 ( \2+  (1-A)2)  + A(l-A)  + (3)  A(l-X) 
3(X2+  (1-A)2)  + 2 A(l-  X)  + 4 A(l-  X) 


Thus  C(P*)  = — . Now  consider  the  following  realization: 

p(0)=P2,  p(l)=Plf  p(2)=P2,  P(3)=P1,  ... 

Then  x (T)  is  deterministic,  x(0)  = 7r1>  x(l)  = 7r ■ , x(2)  = tt^,  and 

, . T c (x  (t) ) 

3 = ?(P  ) > lim  inf  J = - 

T-x»  t=l 

and  ?(P*)  is  not  a lower  bound. 

We  turn  now  to  the  proof  of  the  theorem. 


Proof  of  Bounding  Theorem: 


Step  1.  Let  uT  be  a finite  string  of  characters  from  X.  Let  n^ 
be  the  number  of  character  i in  u>T,  assume  that  n^>0  i=l,2,...,N. 

Let  n„  be  the  number  of  occurrances  of  the  pairs  x^x^  (including 
the  possibility  that  xi  is  the  last  element  of  ojT  and  x^  is  the  first.) 
Since  n^O  we  can  define  the  vectors 


Let  P be  the  matrix  (P^  P2 


. PM).  We 
N 


claim  that  Pp  = p has  a solution  p with  up  = 1. 

Example . Let  ^=1132311311231331.  Then  n1=8, 

n_=2,  n =6.  Write  n. . into  a matrix: 

13 


lnij)  = 


, and  then  P = 


Now  notice  that  just  given  P and  the  length  of  (i.e.,  16), 

n^,  n2  and  n3  can  be  determined.  Moreover,  jl,  can  be 

determined  just  from  P.  This  is  because 


nl 

f n 

n„  — 

n 

2 — 

n. 

n 

3 

nl  + n2  + n3  = 16 


can  be  solved  for  n^,  n2  and  n^  uniquely. 

Proof : Follows  because  P is  an  ergodic  stochastic  matrix  (see 
Section  2.9) 

The  significance  of  this  observation  is  the  following.  Let 
>-  ? r (x  (t) ) 


- 2 ^ 
T t=i 


Then  5 is  a function  of  P , in  fact  sT  = s(P).  What  we  will  do 
next  is  show  that  P-*-  Pe  (P  as  T-^°° 

Step  2.  Let  ^ be  a stochastic  process  in  a.  and  let  u be  a 
well-tempered  outcome.  Let  z^(t)  be  the  indicator  of  x^  as  discussed 
earlier.  Then  for  u , and  all  i=l,2,...,N, 


l zi(t)  = 

t=0 


Proof:  Let  D be  the  set  of  i for  which  the  sum  does  not  go  to 
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infinity,  i.e.,  for  which  the  sum  is  bounded  above.  Now  since 
each  z^(t)  is  either  one  or  zero,  there  must  be  some  T\  for 
each  i€D  such  that  z^(t)=0  when  t T^.  Let  T be  the  largest  of 

V 

Let  d€D.  For  every  Pift,  there  is  a strictly  positive  prob- 
ability P(i,d)  for  each  i.  Let  i ^ D be  fixed.  Since  C V is  compact, 
there  must  be  some  strictly  positive  minimum  for  P(i,d)  over  ft , 
call  it  p.  Now 


lim 

n -*■<*> 


^ 2d^Tk^  " 

2i(Tk>=1 
Tjc>  T 

k ^ n 


zi(Tk)=1 

Tk>T 
k <n 


w - p 

n 


lim  -2-  = ~P  since  z^  =0  for  T^>  T.  So  clearly  we 

n-»»o 

have  a contradiction  as  the  above  limit  is  required  to  go  to  zero  for 
any  well-tempered  outcome 


Step  3.  Let  1*1  be  the  absolute  value  of  the  largest  element  in 
4.  By  | fi-  P J we  mean 

inf  | P'-p| 
p’e/» 

which  will  be  zero  iff  P t ft,  since  ft  is  compact.  We  want  to  prove 
that  if  P is  close  enough  to  ft  in 


The  second  part,  of  the  theorem  is  a direct  consequence  of  the  argument ' in 
2.7. 


3.1  Balancing  an  Inverted  Pendulum 

In  the  previous  Chapter,  ambiguous  processes  were  de- 
fined for  a given  plant  and  compensator  by  means  of  a concep- 
tualization II  and  transition  bound  F.  In  this  Chapter  we  will 
solidify  the  ideas  presented  so  far  by  bounding  the  perfor- 
mance of  a multi-state  compensator  for  an  inverted  pendulum. 
This  example  will  also  serve  as  a vehicle  for  introducing 
simplex  measure  spaces,  a special  kind  of  ambiguous  measure 
space  on  which  the  transition  bound  F is  easily  defined.  We 
will  formulate  the  inverted  pendulum  first  as  a deterministic 
system,  analyze  its  performance,  and  then  introduce  uncertain- 
ty modeled  in  a simplex  measure  space  and  show  that  this  un- 
certain plant  can  be  analyzed  using  exactly  the  same  methods 
as  the  deterministic  case.  The  analysis  of  probablistic  un- 
certainty, would,  clearly  be  much  more  difficult,  strengthen1- 
ing  our  claims  on  the  utility  of  ambiguous  measure  spaces  as 
models  of  uncertainty. 

Consider  the  inverted  pendulum  with  a torque  applying 
actuator  illustrated  in  Figure  (3.1).  Let  9 (T) , 8 (T) , u (T) , m 
and  1 be  the  pendulum  angle,  angular  velocity,  control  torque, 


mass  and  length  respectively.  The  linearized  differential 
equation  for  this  system  is  then: 


or  xk+l  = ^k + Buk 

where  a = /g/i, . Relating  this  back  to  the  notation  of  Section 

2 

2.3,  we  take  the  state  space  X=  l,f:XxU+Xas  given 
in  the  above  equation,  and  the  control  torque  is  restricted  to 
the  range  -5  to  5 kgm2/52,  so  U = [-5,5]. 

The  control  system  for  this  pendulum  is  very  simple, 
there  are  sensors  for  both  9 and  0 , but  the  only  processing 
which  is  performed  on  these  two  sensor  voltages  is  to  compare 
each  one  with  two  pre-selected  levels,  as  shown  in  Figure  (3.2). 

Letting  -9^  and  9^  be  the  threshold  levels  for  0,  and  -02 

• • 

and  02  be  the  threshold  levels  for  0,  we  see  that  there  are 

only  nine  possible  outputs  from  this  coder,  call  them 
Y = where,  for  example,  y^  might  correspond  to: 

0 e (-oo,-01) 

y4  ^ • 

9 e ( -0 2 , 0 2 ) 

Thus  we  have  a coder  and  the  mapping  C:X->- Y is  shown  dia- 
grammatically  in  Figure  (3.3) . The  actual  values  of  the 
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thresholds  are  also  shown. 

The  compensator  has  six  states,  so  let  Z = {z. , . . . ,z.} . 

-L  D 

The  state  transition  mapping  and  readout  map  of  the  compensa- 
tor are  diagrammed  in  Figure  (3.4).  To  complete  the  description 
of  the  closed-loop  system,  the  initial  state  of  the  pendulum, 

x(0),  is  known  to  lie  in  C 1(y5),  and  z(0)  = z^.  In  our  pre- 

2 

vious  terminology,  we  have  nQ  = The  performance  in- 

dex or  cost  associated  with  this  compensated  system,  J,  is  de- 
fined by 


R(x)  4 02  + e2  (See  sect.  2.7.) 

To  arrive  at  good  bounds  on  J we  must  find  an  appropriate 
conceptualization  II  and  transition  bound  F.  We  concentrate 
first  on  finding  a suitable  II. 

The  smallest  II  is  just  the  set  Y x Z;  so  we  will  try  first 
the  conceptualism 
II1  = Y x Z 

and  see  what  kinds  of  transition  bounds  result.  Consider 
H.  If  y ^ y5,  then 

r(y)  = sup  { R (x) I xeC-1(y)>  = » 


This  means  that  the  upper  bound  on  J will  be  00  if  x(t)  even 

steps  once  outside  of  area  5.  In  fact,  x(l)  could  be  outside 

area  5 with  probability  one;  consider  y = 5 Q which  is 

l coy) 

.201 

certainly  in  ^ 5 , , but  then  y.  = 5 ..  , which  is  in 

C i(s)  1 ( ^ ) 

C (y3)  with  probability  one.  On  foe  basis  of  this  information 
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we  can  conclude  that  either  II ^ is  not  a very  good  conceptuali- 
zation for  deriving  bounds,  or  that  the  pendulum  is  not  stabi- 
lzied  and  J = 00 . We  investigate  further  with  a more  sophisti- 
cated II;  clearly  the  question  is  how  to  augment  II  sensibly. 

To  do  this  we  will  compute  some  "typical"  histories  of 
the  closed  loop  system,  starting  from  a "stabilized"  position, 
through  an  unstable  period  when  corrective  control  is  applied, 
and  hopefully  back  once  again  to  a "stabilized"  position  with- 
in the  bounds  in  which  the  sequence  began.  An  obvious  candi- 
date for  a "stable"  area  of  X is  C ^(s),  since  it  is  the  only 
area  with  bounded  cost.  Looking  over  the  compensator  stable 
space,  we  see  that  in  states  z^  and  z^.,  no  corrective  control 
is  applied  when  Cx  = y^,  so  we  could  take  either  of  these  to 
be  the  corresponding  "stable"  state  in  the  compensator;  we 
choose  Zy  Then  we  now  wish  to  examine  "typical"  histories 

which  began  in  the  "stable"  closed-loop  state  (^5)  . 

Z1 

Notice  that  C 1(s)  is  convex,  so  if  C(x(T))  = y,.,  then 


x(T)e  cvx 


If  also  u (T)  = 0,  then  x(T  + 1)  must  also  lie  in  a convex  re- 
gion X,  since  f is  a linear  map.  In  fact. 


(T) +1  e cvx 
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= cvx 


for  the  particular  value  of  A given  earlier.  This  convex 


region  is  shown  in  Figure  (3.5).  Let  us  relate  this  back  to 

fy5l 

the  terminology  of  Chapter  2.  If  II.  = 

J.  z, 


then 


2 2 

= p -1  5 = * -1 
C ( 5)  C i(5) 


What  we  claim  above  is  that 


FoV 


= p £ 

0*  -1 


(5) 


= 


AC_1(5) 


is  indeed  a transition  bound  (where  AC-1 (5)  indicates  the  set 
of  pointwise  multiplications  {Ax;xe  C-1(5)}).  Earlier  we 
called  AC  *(5)  the  infimal  support  of  F^^  . Let  us  postpone 
the  discussion  of  whether  or  not  F is  a transition  bound  and 
get  on  with  the  computation  of  "typical  histories". 

So  far  we  have  calculated  an  infimal  support  for  x(T  + 1) 
when  tt  (T)  = it  . Evidently  x (T  + 1)  could  be  in  several  coder 
areas:  y,. *y 3, y^ # y7 , y8 . Each  of  these  possibilities  is  the  be- 
ginning of  a different  history.  If,  for  example,  Cx(T  + 1)  = 
y5,  then  we  have  "returned"  to  the  stable  state,  tt  , from  which 
we  began,  and  can  consider  this  history  as  finished.  If,  how- 
ever, C(x(T  + 1)  = yg,  for  example,  we  have  not  yet  returned 
back  to  the  stabl  » history  tt^,  so  we  note  this  as  a possibility 
which  must  be  explored  further.  Hence  we  increase  our  "concep- 
tualization" of  the  closed-loop  systen.  by  adding  the  history: 


*6 

Z1 
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( z (T  + 1)  = z1  because  T(y5,z1)  = z^.).  Thus  let 
n2  = niU  11 2 


We  can  compute  ; it  is  just 

2 


The  intersection  of  these  two  simplexes  can  be  found  using 
analytical  geometry  (actually  C ^(6)  is  not  a simplex  proper 
because  A is  unbounded,  but  it  is  still  defined  by  a finite 
number  of  hyperplanes.)  Now  the  reason  we  took  the  coder  equiv- 
alence areas  to  be  open  is  that  this  type  of  intersection  can 
often  be  a single  point  or  line  segment,  and  such  theoretical 
uglies  (and  physically  meaningless  possibilities)  can  be 
avoided  when  the  coder  areas  are  open.  Thus  we  will  always 
interpret  convex  sets  as  open. 

Anyway,  we  must  continue  our  investigation  of  the  possible 
histories  starting  now  with  tt2-  Hopefully  all  of  the  possibi- 
lities  eventually  return  to  y5  , or  at  least  somewhere  in 
_1 

C (5).  If,  however,  there  is  some  sequence  of  histories 
which  go  off  to  infinity,  all  having  non-zero  probability, 
then  we  can  conclude  the  system  is  not  stabilized.  In  our 
example  here  this  doesn't  happen  (as  the  reader  would  expect), 
and  all  histories  do  indeed  settle  right  back  down  into 
Let  us  take  (tt1,tt2)  a step  further: 


*5 

Z1 


Note  that  all  of  our  histories  begin  in  y5  , so  it  is  really 

lzlJ 

unnecessary  to  write  each  compensator  state  in  the  history, 
i.e.  , 


could  be  written,  with  no  loss,  as 
(5),  (65  ),  (565  ) 

since  the  successive  z(T)'s  can  be  computed  with  t.  Thus  a 
next  history  in  our  sequence  might  be 
n = ?2 

(565)  ( (A(AC‘1(5)AC‘1(6) ) - 5B)AC_1(5) )-3B 

which  is  very  explicit,  to  demonstrate  that  the  n's  are  in  fact 
finitely  computable  and  representable  quantities.  We  won't 

write  any  more  out,  but  show  the  convex  regions  with  pictures. 

fv  'l 

Our  typical  sequence,  starting  and  ending  in  J 5 , is  shown  in 

lzlJ 

Figure  (3.6).  Since  every  n is  a collection  of  measures,  this 
means  that 

n (55565)Cn  (5) 

and 

Z (55565)  = Z (5)=  Z1 

Another  such  sequence  is  { (5) , (25) , (525) , (5525) , 


(55525)}.  This  is  illustrated  in  Figure  (3.7). 
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After  having  computed  all  the  possibilities,  we  get  the 
diagram  shown  in  Figure (3. 8).  Each  circle  indicates  a parti- 
cular history,  and  the  arrows  show  transitions  that  are 
possible  (technically,  transitions  of  the  journal  state) . By 
adding  all  these  histories  to  11^,  we  have  a conceptualization 
from  which  definite  bounds  can  be  obtained.  Therefore  take 


H = nxU  { (25)  , (35) , (65)  , (525)  , (535)  , (635) , (565) , (665)  , 

(865)  , (5525)  , (5535) , (5635)  , (5565)  , (8565)  , (5665)  , (5865)  , 
(55525) , (55535) , (55635) , (58565) , (55665)  , (55865) }u{. . .} 
where  (...)  represent  the  histories  obtained  symmetrically  from 
(45)  , (75)  and  (85) . 


Having  II,  to  define  the  corresponding  ambiguous  process 
all  we  need  is  to  compute  In  this  case  it  is  quite  easy 

to  do  so.  For  we  know  that  for  any  history  7reH, 


% * 

where  S is  some  simplex.  Thus  as  long  as  x e S , 5 e £c,  and 

X D 

if  an  arrow  exists  between  tt  and  tt^,  this  means  just  that 


tion  shown  can  be  as  high  as  one.  If  there  is  only  one  arrow 
leaving  it,  then  the  probability  of  this  transition  must  be 
one.  Otherwise  each  probability  could  be  anywhere  between  zero 
or  one.  A typical  would  look  either  like  this: 


0 

[0,1] 


or  this:  [0,1] 

0 


Relying  on  the  analysis  of  ambiguous  processes  presented 
in  the  last  Chapter,  we  know  that  J can  be  bounded  by  choosing 
an  extreme  point  P e P such  that  £ (P)  is  just  a matrix  with  a 
"one"  in  every  column,  so  we  are  looking  for  the  sequence  start- 
ing and  ending  with  (s)  with  highest  average  cost.  It  is  shown 
in  Figure  (3.9).  Thus 


^(P*}  = s [r(lT(5))  + r(’r(25))  + **•  + r(ir  (58525)  ) j = *007 
and  the  root-mean-square  error  in  x(T)  must  be  less  than  /j, 
or  less  than  .085.  This  completes  our  analysis  of  the  deter- 
ministic system. 

3.2  The  Noisy  Inverted  Pendulum 

Our  model  of  the  pendulum  has  been  somewhat  unrealistic  be- 
cause we  have  not  included  any  uncertainty  in  its  behaviour 
due  to  random  disturbances.  We  will  show  that  noise  can  be 
included  in  the  model  very  easily,  and  that  our  method  of 
analysis  can  still  be  carried  through.  In  fact,  the  represen- 
tation of  the  measure  sets  and  transition  ranges  P^  are  a 
direct  extension  of  the  convex-set  representations  of  the 
previous  section,  when  the  noise  model  nu  is  a simplex  measure 
set. 


w 
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Suppose  our  hypothetical  inverted  pendulum  were  out-of- 

doors,  subjected  to  random  breezes  and  gusts.  For  simplicity 

assume  that  the  magnitude  and  direction  of  these  gusts  change 

fast  enough  so  that  the  disturbances  over  each  period  of  time 

(0  , A ) , [A , 2A ) , ...are  independent.  Breezes  are  more  common 

than  gusts,  let  us  say  a factor  of  about  100.  A typical 

breeze  applies  a torque  on  the  pendulum  of  no  larger  than  .2 
2 2 

Kg  m /s  with  unknown ‘direction.  Gusts,  on  the  other  hand, 

2 2 

can  apply  torques  as  large  as  1 Kg  m /s  in  any  direction. 
Symbolically  we  are  saying  that  V(T),  the  random  torque  applied 
between  TA  and  (T+1)A,  has  measure  in  £ 2 2)  ninetY-nine 

percent  of  the  time,  and  in  £ ^_^  jj  on  percent  of  the  time. 

Of  course  this  is  equivalent  to  saying  that  the  measure  of 
V(T)  is  .01  times  some  measure  in  C (_1  -jj  plus  .99  times  some 
measure  in  £ 2 2)  * This  is  an  example  of  a simplex  measure 

set. 

Let  us  adopt  a standard  notation  for  simplex  measure  sets. 
The  three  simplexes  in  3R  in  question  are: 
sL  = [-1 , - . 2] , s2  = [-.2, .2] , [.2,1] . 

Making  additional  assumption  that  a gust  is  equally  likely  in 
either  direction  we  write 

n = . 005£  + .99£  + .005? 

1 S2  s2 

(In  our  original  description  £o  and  £ would  not  have  had 

•’i  S2 

definite  weights,  and  although  this  can  certainly  be  represented 
in  a more  general  notation,  we  choose  to  keep  this  discussion 
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as  simple  as  possible.  Also,  we  could  have  taken 
n = .015  + .99?  , but  then  s.us,  is  not  a simplex 

V S^US2  S2  -!■  “ 

(and  not  convex) . ) 


ry6'\  1^51 

Let  tt z1  as  before,  and  let  us  calculate 
and  P^.  Since 

^tt  = ^ -1  F0  P -1  ^ 

*1  C 1 (6)  u C ±(5) 

Let  us  look  at: 

v -i 

0 C 1(5) 

From  the  description  above,  we  have  that 
f(x,u,v,)  = Ax  + Bu  + Bv 

since  v(T)  is  just  an  additive  torque.  What  we  require  of  F 
(from  section  2.6)  is  that: 

FnC  1 0 (y:y  = F (y1,  A(T)),y1e  c ,X(T)e  n ) 

U C-1(s)  U C_1  ( 5 ) v 

Now  Fo(y^,  X (T) ) is  the  measure  of  Ax  + Bv  when  the  measure 

of  x is  y^  and  the  measure  of  v is  X(T) . Since  y^e  £ . , 

C_J-(5) 


the  measure  of  Ax  must  be  in  £ , . Since  X(T)e  n , then 

AC-1  (5)  v 


the  measure  of  Bv  must  be  in 

.OOSCbs,  + .9^bs2  + •005?Bs3 

We  ask  that  the  reader  check  again  that  there  is  no  B.S.  in 
this  equation. 


5 , * ( . 005y , + . 99y  - + .005y,  = .0055  , y,  + .99?  - y- 

C_1(5)  1 2 3 C_1(5)  C_i(5) 

+ .0055  i y.,  since  convolution  is  linear,  and  from  our  pre- 
C-1(5)  J 

vious  reasoning  we  have 

5 -1  e ^ -1 

AC  (5)  AC  (5)+Bs. 


and  so 


5 i * ( . 0055o  + .995B<,  + -0055b_  ) 

AC-1 (5)  BS1  Bs2  Bs3 

0 . 0055  _!  + .995  _!  + .0055  , 

no  ^ / r \ i n _ o ^ / P \ i n no  ^ 


AC  (5)+Bs. 


C ( 5 ) +Bs . 


AC  ( 5 ) +Bs . 


» \ w / ■ wiJ  2 nv  \ w / ■ 

We  can  then  take  the  right  side  of  the  above  equation  as  the  trans- 
ition bound  Fn5  n • This  transition  bound  is  fairly  weak, 

0 C_1(5) 

because  it  introduces  a lot  of  extraneous  measures,  but  it  is 
simple  to  compute.  Notice  that  since  both  C 1 ( 5 ) and  Bs1  are 
simplexes,  the  sum  is  also  a simplex.  The  three  simplexes 
corresponding  to  the  three  simplexed  3^,82,83  are  shown  in 
Figure  (3.10) . 


- . 


T 
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Now  we  can  finish  the  computation  of  n : 

3 


n =P  , ( . 0055  , + .995  . + 

1 C (6)  AC-X(5)+Bs1  AC  X(5)+Bs2 


.005?  , 

AC  (5)+Bs3 


n = .0055  , , + .995  _! 

*1  AC  1(5)+Bs1)AC  x (6)  (AC  X(5)+BS2)AC  •L(6) 


.0055  -I  n 

( AC~ X ( 5 ) +Bs 3 ) AC  x (6) 


Let  us  compute  5 _i  _1 

(C-1  (5) +Bs1) AC-  (6) , for  example. 


AC-1  (5)  = cvx 


.13 

.4 


.07 

-.21  ' 


-.07 

.2  1 ' 


-.13 

-.4 


Bs^  = Bcvx  {-l,-.2} 

f-.005l 

-.1 


= cvx 


! 

i ■ cv*  | 


AC_1 (5)  + Bs 


C ^(6) A (AC  ^(5)+BS^)=  cvx 
C-1(6)A(AC-1(5)+Bs2)=  cvx 
C~1(6)A(AC"1(5)+Bs3)  = cvx 


The  above  three  convex  sets  specify  n . To  find  P. , we 

^1  x 

first  compute 

9 -i  _i  f _ n 

c 1(l),...C  1 (9)  -5 


‘-4-  - 


■— 
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F_5(.005Ct  + .99CT2  + .005CT3  = 

'•005EAT1-5B+-995AT,-5B+-0055AT,-5B»‘(-0055BS,+-99;B  +'0055Bs  > 

1 2 3 1 S2  3 

* EPjk5jk  ”here  5jk  * 5(AT.-5B)+Bs  Pjk  is  prob’  =°e«i=ient. 

3 K 

In  actually  performing  this  type  of  analysis  on  the  com- 
puter, one  would  like  to  keep  the  number  of  simplexes  bounded, 
for  if  we  were  to  continue  with  the  present  method,  the  suc- 
cessive histories  would  have  an  exponentially  growing  number 
of  terms.  We  mention  here  that  it  is  possible,  with  only  slight 
weakening  of  the  bounds,  to  keep  the  number  of  terms  in  each 
^constant.  Roughly,  this  can  be  done  by  finding  some  "typical" 
supports  of  the  simplex  measure  sets,  including  a support  which 
is  large  enough  to  contain  each,  and  then  grouping  the  simplex 
measures  sets,  including  a support  which  is  large  enough  to 
contain  each,  and  then  grouping  the  simplex  measure  sets  into 
the  "typical"  sets. 

Now  to  find  a bound  for  HP^,  notice  that 


C"1(l) , 


C (T  . n)  x (T  . n_)x...x(?  . n ) 


(9) 


(1) 


(2) 


(9) 


Also  notice  that 


is  either  [0] , [1] , or  [0,1]  where  £-k  = ?AT._5B)+  B » since 


if  C 1 ( i ) CinfsP^,  (the  infimal  support  of  ATj-5B+Bsk)  T = 1; 

if  C ^ (i) Ainf sP 0,  T = 0,  and  otherwise  any  probablity  between 
one  and  zero  is  possible,  since  the  unit  mass  is  a measure  in 
and  the  unit  mass  can  be  taken  in  the  infimal  support. 
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Thus  all  we  need  are  the  intersections  of  each  of  the  nine 
supports  (AT.-5B)  + Bs,  with  each  of  the  nine  areas  C 1(i). 

When  this  computation  is  made,  it  is  found  that  all  nine  of 
the  supports  either  all  intersect  or  all  do  not  inter- 

sect each  C-1(i);  for  example,  all  supports  intersect  C 1(5). 
Thus 

t , n=2  p ..  ? . 

C~1(5)  77  j=l , 3 3k  C~1(5)  3k 

k=l , 3 

= I P.,V[0,1]  = t0'1^ 

j,k  3k 

So  P,-  is  the  same  for  the  noisy  pendulum  as  for  the  determin- 
istic one.  I have  not  actually  calculated  P for  the  noisy 
pendulum  but  each  P^  can  be  calculated  as  we  have  calculated 
P^.  The  analysis  of  the  noisy  pendulum  follows  exactly  the 
same  lines  as  the  analysis  of  the  deterministic,  except  for 
the  minor  change  that  the  are  no  longer  representable 

by  single  convex  support  sets,  but  instead  require  a collection 
of  support  sets  and  weights.  It  is  clear  that  simplex  measure 
sets  are  well-suited  to  the  type  of  analysis  of  finite-state 
compensation  systems  we  have  outlined  here. 

3.3  Simplex  Measure  Sets 

In  the  previous  sections  we  motivated  the  simplex  measure 
sets  in  the  context  of  balancing  an  inverted  pendulum.  In 
this  section  we  formalize  the  notions  and  will  find  a tighter 
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transition  bound  F for  linear  systems  than  the  one  used  pre- 
viously. This  material  is , however, somewhat  technical. 

Consider,  from  Example  (3),  section  2.2.,  the  measure  set 
n , where 

np  = { |i  | y ( — 00 , 0 = 1-p,  (0,°°)  = p) 

In  the  notation  of  the  previous  section  we  would  write: 


np  (1_p)  C(-®,0)  + p^(0,®) 

Suppose  we  wish  to  bound  the  set  n *n  . Using  the  method  of 

P P 

the  previous  section,  we  would  take: 

np*I1pC  (1"P>  5 (-.,0)  + <->,0)  + 2P  <i-p)  5 (_„r0)  + (0,„)+P25  (0, .)  + (,. 

■ (1-p)2*<-,0>  + ♦ P2  5(0,.) 

As  an  example  of  a measure  that  is  in  the  right-hand  measure 
set  but  not  in  the  left  hand  set,  consider 

Vi  = (1-p)2  6_1  + (l-(l-p)2)51 

To  see  that  y is  not  in  n *n  , first  observe  that  due  to 

P P 

the  positivity  constraint,  if  n *n  then  there  must  be  a 

P P 

y i = a]_^x  + 6i^y  and  ^2  = a2^x  + 625y  such  that 


1) 


x1x2  e (-“,0,) 


2)  y2y2  e (0,®) 

si  + y2  = -i 

X1  * x2  = 1 

yjL  + x2  = 1 

yx  + y2  = 


1 


*T  
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4)  axa2  = ( 1-p) 2 , a-Lb2  + b^  + b1b2  = 1 - (1-p)2 

5)  aL>0,  a2>0,  b^O,  b2>0 

i.e.,  y1*U2  = V • However,  there  is  no  solution  to  the  above 
condition  for  x,y,a,  and  b. 

This  suggests  an  alternate  description  of  simplex  measure 
sets,  which  can  exclude  the  possibility  shown  above. 

Suppose  we  describe  the  convex  set  n from  Example  (1) 
in  terms  of  s set  of  "extreme  points."  The  set 

Sp  ~ ^x1x2  I x1<0  ,x2-0} 


where  6 


is  the  measure  on  R with  mass  (1-p)  at  x.  and  mass 
lx2  1 


p at  x2 , could  be  considered  the  extreme  points  of  rip.  Certain- 
ly the  convex  hull  of  s is  in  n , but  to  get  equality  we  must 
consider  an  extended  kind  of  convex  combination. 


Notice  first  that  the  set  s is  naturally  isomorphic  to 

2 fxll 

a set  s = (-°°,0)  x (0,°°)  C 3R  , where  we  associate  xl  e s 
A ^ z x 


with  6 . This  enables  us  to  define  a "continuous"  convexi- 

X1X2 

ty  of  the  elements  in  Sp  by  means  of  a positive  measure  p on 

JR  with  unit  mass  in  s ; 

x* 


5 dp 
Sx  X1X2 


represents  a measure  on  R defined  by 


5x  x dp' 
s xlx2  > 
x 


(B)  = 


5 (B)  dp,  BeB 

sx  X1X2 


i.  - ~- 


I 


Since  p (sx)  = 1 the  resulting  measure  will  be  a probability 
measure,  and  furthermore 


% = 


sx5*lx2dP  P>°,P(S*)  = 1 


This  representation  of  np  is  a "simplex"  representation 
because  the  measure  set  is  completely  specified  by  the  simplex 
sx  and  mass  vector  j*’^]  for  x.,  and  x? . The  advantage  of  using 


this  somewhat  cumbersome  representation  is  the  ease  in  bounding 
the  set  of  pairwise  convolutions  of  measures  in  n , for  we  see 
that  r 


VnP  = 


x=  1 es 


xlx2 


•S  1 1 dp  p(s  ) = 1 
lxlx2  1 x , . 

xil  e <sx>-  1 


x =11  es 


^ ^xlx2  ^xlx2 


xes  x es 
P P 


Now  let  6 


v v v v be  the  measure  with  mass  (1-p) 1 at  y. , (1-p) ; 
Y1Y2Y3Y4  1 


at  Y2'Y2  and  P at  ^4'  so  that 


6xlX2*5xl  x2  5 (xl+x2  ) (xi+x2  ) (x2+x1  ) (X2+X21) 

Given  xes  and  x^es  the  possible  values  for 

X X 

y = (y^^^V  = ( (x^^+x^^1)  (x1+x21)  (x2+x11)  (x^x^)  ) are  in  the 


\ xl+xl 

J x1+x21 

) *2+xl1j 

(.  lx2+x21i 


rXl'i  rx,  l . 

! N^x'lxj1]6^  4 +‘(sx'sx> 


L, 


set : 


r 


which  is  the  same  set  as: 


W1  0 + w2 


+ w + w 

-1  W3  1 4 0 


Wl-W4e  f0'00^ 


which  is  again  a simplex.  Thus  n *n  is  bounded  by  another 

P P 

simplex  measure  set. 

rp*np  C f VP  p(+*(sx'sxn  = 1 \ 
i +*<Vsx}  J 

It  is  clear  from  the  way  we  defined  +*(sx,sx),  that  the  measure 
2 2 

y =(l-p)  + (l-(l-p)  5^  is  not  in  the  right-hand  set,  so 

this  method  of  bounding  convolutions  is  superior  to  the  one 
used  in  the  previous  sections. 

The  above  comments  are  mentioned  incidentally  in  this 
thesis,  so  we  will  not  go  through  a completely  formal  develop- 
ment. But  some  generalization  is  in  order. 

Definition . A simplex  sC^n  is  a mass  simplex  if  for  every 
seS,  s-0  and  (1  1 ...  l)s  = 1. 

Definition.  (x,m)  is  an  (n,k)  vector-mass  pair  if  x is 

k 

a real  n by  k matrix  and  meR  is  an  element  of  a mass  simplex. 

Notation.  Let  (x,m)  be  an  (n,k)  vector-mass  pair  and  let 

x by  the  matrix  of  columns  x^x2, . . . ,xk,  and  let  m = (it^ , . . . ,11^)  T . 

We  denote  the  measure  on  Rn  with  mass  nu  at  x1,  i=l,...,k,  by 

5 , . . 

(x,m) 

Definition . Let  n be  a measure  set  on  Rn.  n is  a simplex 


simplex  smCR  such  that 


n = < u:y  = 


s xsm 
x m 


5 (x,m)dpda'p (sx)=1'a (sm)=l,p-0,a-0 


the  integral  is  defined  as  the  measure 


s xs 
x m 


6(x,m)dpda  <B>  = 


s xs 
x m 


S(x,m)(B)  BeB 


If  n is  a simplex  measure  set  we  write  n = cvx  (s  ,s  ) , since 

x m 

sx  and  sm  are  t°r  each  simplex  measure  set. 

In  addition,  every  pair  (s  ,sj yields  a measure  set,  and 

X lU. 

we  define  the  simplex  measure  space  on  Rn,^n,  to  be  the  set 
of  all  simplex  measure  sets  on  Rn. 

j2n  also  satisfies  all  the  axioms  of  an  ambigous  measure 
space,  and  the  only  difficulty  in  showing  this  is  showing  that 
ps (y)  £ j8n  where  s is  representable  and  n£^n  . We  will  omit 
the  proof  here. 


3.4.  Topics  for  Future  Research 
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In  this  thesis  we  have  shown,  on  a rigorous  basis,  how  ambiguous 
processes  can  be  used  to  analyse  the  performance  of  complex  nonlinear 
feedback  systems . In  the  past  sections  we  have  shown  how  a conceptuali- 
zation TT  can  be  methodically  computed,  and  how  a transition  bound 
arises  naturally  for  linear  systems  with  simplical  measure  set  un- 
certainty, and  these  two  objects  then  define  an  ambiguous  process  from 
which  performance  bound  can  be  obtained.  The  next  step,  a possible 
topic  for  future  research,  is  to  precipitate  an  actual  bounding 
algorithm  for  finite-state  compensated  systems. 

Another  very  interesting  topic  for  future  research  is  a design 
methodology  based  on  ambiguous  processes.  Our  preliminary  results  in 
this  area  were  too  scanty  for  presentation  here,  but  it  seems  that 
memoryless  compensators  with  a fixed  coder  and  fixed  plant  have 
sufficient  structure  that  optimization  can  be  performed.  Unfortunately, 
the  design  of  inite-state  compensators  with  memory  seems  to  be  a very 
different,  and  a very  important  problem  for  future  research,  because 
the  possible  set  of  observations  depends  on  the  compensator' s memory 
structure , a situation  which  does  not  arise  in  the  memoryless  case  or 
in  classical  control.  However,  it  is  also  likely  that  in  such  complex 
design  problems,  general  theoretical  simplifications  can  only  be  made 
to  a limited  extent,  and  design  must  rest  on  searching  strategies 
(as  in  stochastic  control.)  Still  the  end  results  would  be  directly 
implementable  controllers  with  very  definite  performance  bounds,  and 
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so  the  design  of  compensators  using  ambiguous  processes  as  a methodology 
would  certainly  seem  an  interesting  problem  for  future  research. 

|' 

i 
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